-- Logs begin at Wed 2022-07-20 14:46:41 UTC, end at Wed 2022-07-20 16:04:55 UTC. -- Jul 20 15:22:11.375144 dv-cinder-848759-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 15:22:11.383715 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 15:22:11 2022] *** Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: nodename: dv-cinder-848759-3-iscsi Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: machine: x86_64 Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: clock source: unix Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: pcre jit disabled Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: detected number of CPU cores: 4 Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: current working directory: / Jul 20 15:22:11.385727 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: detected binary path: /usr/bin/uwsgi-core Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: your processes number limit is 47680 Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: your memory page size is 4096 bytes Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: detected max file descriptor number: 2048 Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: lock engine: pthread robust mutexes Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: thunder lock: enabled Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 15:22:11.386315 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 15:22:11.430182 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: Python main interpreter initialized at 0x5626280869c0 Jul 20 15:22:11.430182 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: python threads support enabled Jul 20 15:22:11.430363 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: your server socket listen backlog is limited to 100 connections Jul 20 15:22:11.430363 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: your mercy for graceful operations on workers is 90 seconds Jul 20 15:22:11.430536 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 15:22:11.430637 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: *** Operational MODE: preforking *** Jul 20 15:22:11.430928 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: *** uWSGI is running in multiple interpreter mode *** Jul 20 15:22:11.430928 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: spawned uWSGI master process (pid: 80197) Jul 20 15:22:11.431119 dv-cinder-848759-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 15:22:11.432737 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: spawned uWSGI worker 1 (pid: 80207, cores: 1) Jul 20 15:22:11.432737 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: spawned uWSGI worker 2 (pid: 80208, cores: 1) Jul 20 15:22:11.432737 dv-cinder-848759-3-iscsi devstack@keystone.service[80197]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 15:22:16.832598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:22:16.832598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: from cryptography.utils import int_from_bytes Jul 20 15:22:16.833363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:22:16.833363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: from cryptography.utils import int_from_bytes Jul 20 15:22:16.833475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:22:16.833475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jul 20 15:22:16.833870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:22:16.833870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jul 20 15:22:16.882332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 15:22:16.882512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 15:22:16.882705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] command line args: [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 15:22:16.882969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 15:22:16.883142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ================================================================================ {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 15:22:16.883394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] admin_token = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.883635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] config_dir = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.883718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 15:22:16.883855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.883980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 15:22:16.884083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] config_source = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.884163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] command line args: [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 15:22:16.884236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] debug = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.884367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 15:22:16.884456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.884645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ================================================================================ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 15:22:16.885879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.885983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_dir = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_source = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] debug = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_config_append = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.886967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_dir = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_file = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_config_append = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_dir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.887984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.888923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.889963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] publish_errors = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.890941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] publish_errors = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] standard_threads = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.891963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] use_journal = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.892991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] use_json = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] standard_threads = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] use_stderr = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] use_syslog = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.893901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.894014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_journal = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.894131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.894249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_json = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.894362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.894472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_stderr = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.894472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_syslog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.894695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.894813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:22:16.894926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.895877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.896824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.897822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.898950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.899995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.900923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.901949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.902624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.903967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.904981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.905895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.906940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.907958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.908880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.909916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.910936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.911880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.912919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.913961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.914951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.915895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.916931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.917856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.918089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.918210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.918369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.918369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.external = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.919933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.password = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] auth.token = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.920889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.external = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.921932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.password = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.token = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.922979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.923955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.924892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.925941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.926982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.927872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.928931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.929942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.930991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.931959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.932996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.933997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.934916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.935971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.936970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.937972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.938897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.939922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.940854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.941852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.942859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.943923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.944972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.945898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.946958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.947934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.948971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.949909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] role.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] role.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.950972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.951998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.952886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953114 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.953806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.954944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.955890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.956933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.957970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.958996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.959956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.960978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.961908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.connection = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.962960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.963999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.964933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.965046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.965747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.965888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.966949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 15:22:16.967649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.967649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.968207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.968207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.968435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.968617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.968855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.969095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.969333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.969571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.969832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.970083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.970334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.970572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.970821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.971062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.971310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.971542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.971820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.972053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:22:16.972224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 15:22:16.998332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:16.998551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:16.998769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:16.999070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:16.999305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:16.999530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.007682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.008015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.036411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.036638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.037115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.037333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.178953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.180479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.228135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.228367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.228782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.229019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.292578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.292794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.293886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.294101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.317903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.318995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.319127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.319210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.319338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.319549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.330650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.330990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.343613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.343858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.344963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.345093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.345303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:22:17.361802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.362007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:22:17.362203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.362336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.362419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:22:17.362533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:22:17.362737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.362817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.362938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:22:17.363027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.363207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.363286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.363468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.363547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.363724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.364008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.375357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.375626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.375912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.375995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.376190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.376270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.376459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.376744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.384228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.384884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.388036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.388306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.388690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.388776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.388969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.389970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.390830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.391016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.391286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.391558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.416918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.418229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.418534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.418804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.419900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.420178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.424875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.424875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.424875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.427681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.427681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.427681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.432151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.432378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.433315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.433594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.437471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.437752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.438028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.438305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.438576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.438743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.438842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.439888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.440160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.440429 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.440696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.457810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.458178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.459406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.459681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.462639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.464288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.466783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.467071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.468503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.468794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.472098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.473819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.475645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.477483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.479402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.479692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.479995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.480286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.480553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.480886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.481833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.482140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.482431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.482723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.483012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.483299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.500405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.500770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.501035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.501296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.502542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.502917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.503189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.503450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.509141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.510830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.511107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.511389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.513147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.513420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.514178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.514449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.514723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.514996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.515275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.516523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.516802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.517082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.517361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.517688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.529100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.529371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.529645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.529919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.530202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.530487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.530758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.531608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.531913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.532196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.532476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.532756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.533046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.533320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.550355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.553172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.553627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.556483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.556910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.558485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.560356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.560699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.560968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.561248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.561974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.563618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.563920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.564207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.568613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.570358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.571612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.573263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.573570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.573854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.574134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.574402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.577047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.577337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.577620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.577901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.584081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.584363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.584645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.587131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.587416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.587694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.592449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.592723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.592996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.593260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.593522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.593789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.594054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.594320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.594583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.594852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.595120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.595375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.595637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:22:17.595719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.595912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.595992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.596198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.596471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.596740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.597017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.597289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.597576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.597845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.598121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.598395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.598654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.598913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.599185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:22:17.634022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5626280869c0 pid: 80208 (default app) Jul 20 15:22:17.637847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59324-4391-4221-813d-831ecbaae10e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:17.638138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59324-4391-4221-813d-831ecbaae10e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:17.638404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5626280869c0 pid: 80207 (default app) Jul 20 15:22:17.638499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59324-4391-4221-813d-831ecbaae10e None None] PATH_INFO: `/v3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:17.642594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Wed Jul 20 15:22:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:25.267109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:25.267667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:25.267667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:25.272210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:25.277456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:25.278381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:25.278557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:25.346976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: INFO dbcounter [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] Registered counter for database keystone Jul 20 15:22:25.368003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG oslo_db.sqlalchemy.engines [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80208) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 15:22:25.374457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writer thread running {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 15:22:25.552392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG passlib.handlers.bcrypt [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80208) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 15:22:25.552737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG passlib.handlers.bcrypt [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80208) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 15:22:25.606800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:25.608455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:25.730254 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 15:22:25.908015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:25.976871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: INFO dbcounter [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] Registered counter for database keystone Jul 20 15:22:25.996675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG oslo_db.sqlalchemy.engines [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80207) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 15:22:26.002745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writer thread running {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 15:22:26.118760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d9c4-cb85-451b-b546-e16d16204a5e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:26.123236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:26.123923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:26.124208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:26.124480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:26.132480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:26.184478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 15:22:26.184762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2873a-e1ce-47d1-a3f5-3cb39fa0539f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:26.194880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Jul 20 15:22:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:22:26.201509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:26.216391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a2e6b-959e-4013-84b4-a81ea5fbae46 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:26.220815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:26.221928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:26.222218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:26.222498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] PATH_INFO: `/v3/projects/498a38cb72f746aabe0439ba1e952a81` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:26.226599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] RBAC: Authorizing `identity:get_project(project_id=498a38cb72f746aabe0439ba1e952a81)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:26.277215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 15:22:26.278138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcb050-4314-47ea-9d20-2ded37a3af29 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:26.282751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Wed Jul 20 15:22:26 2022] GET /identity/v3/projects/498a38cb72f746aabe0439ba1e952a81 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:28.304797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d4e0-ac64-493e-92db-c554ced7dfac None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:28.305308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d4e0-ac64-493e-92db-c554ced7dfac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:28.305406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d4e0-ac64-493e-92db-c554ced7dfac None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:28.309510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:28.315301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:28.315710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:28.315894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:28.334824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:28.336407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:28.344829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 15:22:28.368560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17d4e0-ac64-493e-92db-c554ced7dfac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:28.380512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17d4e0-ac64-493e-92db-c554ced7dfac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:28.384685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:28.385319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:28.385604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:28.385880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:28.393967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:28.395734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea20f7e-184d-4955-8fc8-702d44e490fa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:28.405099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Jul 20 15:22:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:22:28.410382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:28.419297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b42cef-0cdd-4a5b-aec2-15d8e7c1ee43 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:28.423709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:28.424880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:28.424880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:28.425130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:28.426532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:28.428405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c777b57-cce8-400b-9871-a632032b70c2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:28.429983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Wed Jul 20 15:22:28 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:30.539361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b17e-f217-48ff-a7d2-88d6611c0633 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.539890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b17e-f217-48ff-a7d2-88d6611c0633 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.539990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2b17e-f217-48ff-a7d2-88d6611c0633 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.541303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:30.546502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.546913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.547052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.553193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da03ecd-dc2c-46c4-86f2-5cf246185116 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.553483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da03ecd-dc2c-46c4-86f2-5cf246185116 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.553761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da03ecd-dc2c-46c4-86f2-5cf246185116 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.555048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:22:30.560288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.560642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.560924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.563985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:30.565160 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:30.601304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80207) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 15:22:30.601717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80207) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 15:22:30.645384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:30.647211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:30.648846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:30.669526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:30.693230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:30.749292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:30.771212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:30.772868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0674511-e9c0-4ece-870e-c275eb17f8b9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:30.806142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:30.806929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.807212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.807487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.819856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:30.830994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:30.831932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:30.832734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:22:30.839206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:30.870815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a29d7-4c33-4cdd-946b-eef2dcbf35d6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:30.873771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:30.909277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:30.910170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.910457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.910734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.921382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:30.941093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442d772-c0c0-42e5-aca1-6aee83e4e8cb None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:30.974966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:30.975571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:30.975896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:30.976173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:30.976930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:30.977238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:30.977363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:30.978665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:30.978862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:30.990866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:30.996730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:31.077002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:31.082777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:31.085471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:31.097158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a570c-021f-4b51-a802-8989e9784bc9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:31.132345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:31.133222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:31.133509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:31.133785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:31.135009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:31.136506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:31.138141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:30 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:31.143541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:31.158411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:31.182961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18958e10-06b9-4bcf-bed7-592344d95b71 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:31.217904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:31.218707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:31.218984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:31.219248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:31.237979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:31.239375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:31.242644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbfcd4-61ad-45b3-a9cd-987b2026bc77 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:22:31.262498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:31.263462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:31.264298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:22:31.264298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:22:31.264298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:22:31.264298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:31.264298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:22:31.269533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:31.276671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:31.278016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:31.278306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:31.278584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:31.279808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:31.281281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:31.282889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:31.288676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:31.325200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:31.366800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:31.369416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3f03-7087-4e7f-acee-eac4ef08771f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:31.387925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385db7-18d8-4060-9161-d6cee29dd585 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:31.402687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:31.403344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:31.403626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:31.403931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:31.404652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:31.404956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:31.406337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a0434-4ee3-4792-b40e-c8403e89f1c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:31.421391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:31.422058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:31.422344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:31.422622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:31.423472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.domain.id=default, user.id=4fc861f9b4f146eca38f05771344d34d) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:31.423809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:31.425249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8dfa1e-a439-4639-a569-6ba8a7fff338 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:31.429604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:31.432417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Jul 20 15:22:31 2022] GET /identity/v3/role_assignments?user.id=4fc861f9b4f146eca38f05771344d34d&scope.domain.id=default&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:33.780034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4954-c443-452f-bcec-8512462ae799 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:33.780526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4954-c443-452f-bcec-8512462ae799 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:33.780526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4954-c443-452f-bcec-8512462ae799 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:33.782080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:33.791427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bb1a1-5f03-4344-8cee-6b122c20c86b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:33.792354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bb1a1-5f03-4344-8cee-6b122c20c86b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:33.793391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bb1a1-5f03-4344-8cee-6b122c20c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:33.817435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6c5bb1a1-5f03-4344-8cee-6b122c20c86b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:33.818941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6c5bb1a1-5f03-4344-8cee-6b122c20c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:33.917033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:33.961211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4954-c443-452f-bcec-8512462ae799 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.006208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b8cc-dc11-46b1-b6da-20c33df7da02 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.006852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b8cc-dc11-46b1-b6da-20c33df7da02 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.007321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704b8cc-dc11-46b1-b6da-20c33df7da02 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.009230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:34.014764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.015284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.016148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.055717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:34.058738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:34.189195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4954-c443-452f-bcec-8512462ae799 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:34.210430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 14/25] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:34.228103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:34.244633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.246943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:34.247781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.248070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.248346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.249256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:34.269062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a80767d2-1fdb-45a5-9357-81784b946160 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:34.291849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:34.292761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:34.293937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 12/26] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:34.297459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43b521-8ef8-40ec-a74f-5a6ed529b61e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.297459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43b521-8ef8-40ec-a74f-5a6ed529b61e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.297459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43b521-8ef8-40ec-a74f-5a6ed529b61e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.297459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 13/27] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:34.304909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43b521-8ef8-40ec-a74f-5a6ed529b61e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.453071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a16b1-110c-41bd-8102-c5ceece800de None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:34.485317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43b521-8ef8-40ec-a74f-5a6ed529b61e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:34.489990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:34.490933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.491407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.491937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.493250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:34.494914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9cb77-57ad-4427-8ae1-5d63d3c2fbce None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:34.530176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:34.531250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.531724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.532215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.533224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:34.533737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:34.535415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc11105-0664-4415-8363-e5313dc9ac8b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:34.548494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 14/28] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:34.550337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270b3f-cc10-4177-ae2d-577bc78a2a7c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.550864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270b3f-cc10-4177-ae2d-577bc78a2a7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.551348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270b3f-cc10-4177-ae2d-577bc78a2a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.568984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-98270b3f-cc10-4177-ae2d-577bc78a2a7c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:34.570438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-98270b3f-cc10-4177-ae2d-577bc78a2a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:34.603517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:34.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 15/29] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jul 20 15:22:34 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:22:34.623120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbe88f-3ad3-4dee-b84a-9fc0af190af0 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.623381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbe88f-3ad3-4dee-b84a-9fc0af190af0 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.623747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbe88f-3ad3-4dee-b84a-9fc0af190af0 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.629973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 16/30] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:34.631113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbe88f-3ad3-4dee-b84a-9fc0af190af0 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.658790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 15/31] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:34.660414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.660712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.660991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.677838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:34.679129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:34.724935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:34.782074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbe88f-3ad3-4dee-b84a-9fc0af190af0 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:34.809186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 16/32] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:34.812599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.855079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:34.856046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.856336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.856618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.858420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:34.860063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:34.862027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:34 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:34.862801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:34.939642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:34.945529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e30bc-1d09-4eb3-84e9-92ea0ccd67fa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:34.987946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:34.988673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:34.988955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:34.989236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:34.992569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab41a3-6564-44a5-9ed3-802698fd4755 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.000837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.002313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:22:35.020842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:35.021661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:35.022446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:22:35.022446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:22:35.022446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:22:35.022446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:35.022446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 17/34] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:35.023543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.030927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:35.037347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.038025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.038309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.038588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.039342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.040776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf29574-cbdd-47f3-934f-0942131ddd1a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 18/35] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jul 20 15:22:34 2022] POST /identity/v3/roles => generated 225 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd58-f4e6-4fa7-b02d-4e97f2064d33 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd58-f4e6-4fa7-b02d-4e97f2064d33 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd58-f4e6-4fa7-b02d-4e97f2064d33 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 19/36] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:35.063148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94dd58-f4e6-4fa7-b02d-4e97f2064d33 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.152338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea602de6-d54d-4587-bc42-831df94a328b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.199180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.200057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.200342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.200619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.202160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.203724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600a03a-531c-420b-a31d-244d0ca8df76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.205931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 18/37] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:35.207376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.207710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.208015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.224163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94dd58-f4e6-4fa7-b02d-4e97f2064d33 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.225622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:35.226820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:35.246069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:35.265234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.265916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.266223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.266485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.267226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:35.267536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.270576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.290688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:35.291611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.316807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 19/39] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:35.317417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.358815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:35.411964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5a83-3c9f-4f38-b67c-9a42f23f07c4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.431712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42e82-95a6-4c0f-8c09-fb7c90783909 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.450369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.451249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.451547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.451900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] PATH_INFO: `/v3/domains/default/users/4fc861f9b4f146eca38f05771344d34d/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.454077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4fc861f9b4f146eca38f05771344d34d, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.455583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.470542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 21/40] 10.222.0.27 () {58 vars in 1361 bytes} [Wed Jul 20 15:22:35 2022] PUT /identity/v3/domains/default/users/4fc861f9b4f146eca38f05771344d34d/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:35.471351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.473160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.474206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.474494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.474773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.475505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.477095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa5ca5-4f92-441c-a556-b9b1bb74d1b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.510661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:35.603148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7b735-b391-425e-a3a9-f6943a638886 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:35.618004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:35.641181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 20/41] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:22:35 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:35.643042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f91ca-bd5e-455b-b98c-083f4f8386de None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.643420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f91ca-bd5e-455b-b98c-083f4f8386de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.643792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f91ca-bd5e-455b-b98c-083f4f8386de None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.645187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 21/42] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:35.647535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:35.648473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.648764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.649049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.649830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:35.650880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:35.651181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:35.651271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:35.651470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:35.667633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 22/43] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jul 20 15:22:35 2022] POST /identity/v3/roles => generated 223 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:35.672740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:35.673482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:35.765183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 22/44] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:35.788926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:35.998987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9d9f-40f8-4daa-9661-689f68c92d4d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:36.032850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:36.033581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:36.033871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:36.034153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:36.035340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:36.036829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8eb5a-89de-4f70-8c41-fd2c26a2e8db None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:36.038443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 23/45] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:36.044913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:36.144452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5af21c-0d0a-419a-986a-c7c984f78cf5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:36.173344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:36.179814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:36.180485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:36.180769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:36.181045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:36.181756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:36.183367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681d3da-cebc-4890-9a8b-87ec60d85233 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:36.207537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 23/46] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:22:36 2022] POST /identity/v3/projects => generated 312 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:22:37.579425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9eb6-494f-4728-a972-040dfdf79376 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:37.579949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9eb6-494f-4728-a972-040dfdf79376 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:37.580029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9eb6-494f-4728-a972-040dfdf79376 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:37.581373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 24/47] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:37.586652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:37.587049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:37.587281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:37.604345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:37.605536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:37.671428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 24/48] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:37.694441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9eb6-494f-4728-a972-040dfdf79376 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:37.736404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:37.794977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9eb6-494f-4728-a972-040dfdf79376 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:37.828785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:37.829571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:37.829853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:37.830124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:37.840075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:37.847655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:37.848669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:37.849460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 25/49] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:37.854776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:37.952525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562575f-220b-484e-98a4-a7f26591e840 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:37.966010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:37.988249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:37.988917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:37.989199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:37.989471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:37.990221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:37.990528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:37.991939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.003311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 25/50] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:38.009902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:38.107244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:38.111134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b051acc-10e9-433d-88da-f04d80a75e77 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:38.146652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:38.147519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.147826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.148101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.149501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:38.151016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba70ff5-cdfc-4060-8b2e-bcf2ad15a1a4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.152723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 26/51] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:38 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:38.158147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:38.243315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2017-54f2-4a8a-b056-44fd5d0de562 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.243506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2017-54f2-4a8a-b056-44fd5d0de562 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.243818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2017-54f2-4a8a-b056-44fd5d0de562 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.245227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 27/52] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:38.251730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.252048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.252332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.269607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:38.271141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:38.339793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 28/53] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:38.363233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:38.368544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f8efb-086d-4928-b3d9-41b850558b31 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:38.404947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:38.405742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.406023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.406299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.407470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:38.408945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.410552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 26/54] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:38.416133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:38.460236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:38.463162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:38.468245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf3099-ac27-46f3-bee1-642ad10d12e1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:38.502127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:38.502831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.503117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.503400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.504618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:38.506073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.507689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 29/55] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:38.513136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:38.518258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082f976-2c45-46c3-bf31-cc121729a918 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:38.553436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:38.554103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.554387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.554669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.555513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.domain.id=default, user.id=4fc861f9b4f146eca38f05771344d34d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:38.555853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:38.557296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aa2d1-a3b8-47c0-ae2e-f115d0e4c93d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.566299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 27/56] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Jul 20 15:22:38 2022] GET /identity/v3/role_assignments?user.id=4fc861f9b4f146eca38f05771344d34d&scope.domain.id=default&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:38.614436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e53a2-1a3f-4b03-8802-1e02f16ee5e9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:38.650540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:38.651215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:38.651496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:38.651790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:38.653060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:38.654534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a8ba8-711c-439a-878d-e0451e46f083 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:38.684370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 30/57] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Jul 20 15:22:38 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:22:40.739648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61d27-8912-4730-a652-50dc8c598016 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:40.740165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61d27-8912-4730-a652-50dc8c598016 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:40.740165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b61d27-8912-4730-a652-50dc8c598016 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:40.741496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 28/58] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:40.746877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:40.747273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:40.747451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:40.764033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:40.765227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:40.778553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:40.830863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 31/59] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:40.855882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b61d27-8912-4730-a652-50dc8c598016 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:40.953859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b61d27-8912-4730-a652-50dc8c598016 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:40.960877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:40.987923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:40.988707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:40.988987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:40.989258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:40.990486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:40.991927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08283fda-185f-48a9-9e9b-26a7778e60c1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:40.993783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 29/60] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:40.999220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:41.096513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8000c-9b6b-494b-ad2b-7826b91bca24 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:41.130424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:41.131099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:41.131391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:41.131669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:41.132422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:41.133892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb13f83-b39c-478b-a762-f5c689d82737 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:41.162228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 32/61] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:22:40 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:44.397341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9cf7-dc9a-46cb-9ab6-b466c20d31b4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.397341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9cf7-dc9a-46cb-9ab6-b466c20d31b4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.397341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9cf7-dc9a-46cb-9ab6-b466c20d31b4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.397341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 30/62] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:44.413789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b361-c190-4cce-81c2-d7135be95048 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.414578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b361-c190-4cce-81c2-d7135be95048 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.415055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577b361-c190-4cce-81c2-d7135be95048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.429048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:44.447877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b577b361-c190-4cce-81c2-d7135be95048 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:44.449123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b577b361-c190-4cce-81c2-d7135be95048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:44.546122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 33/63] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:44.594082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9cf7-dc9a-46cb-9ab6-b466c20d31b4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:44.733598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c008ed1-36a8-4365-b630-9127d272308f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.734222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c008ed1-36a8-4365-b630-9127d272308f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.734521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c008ed1-36a8-4365-b630-9127d272308f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.736152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 34/64] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:44.737693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753262-f261-4d47-b6d7-8b414df4987e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.738172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753262-f261-4d47-b6d7-8b414df4987e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.738639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753262-f261-4d47-b6d7-8b414df4987e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.740121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 35/65] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:44.741642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaef275-1dff-465b-9e3c-f885675cfc1f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.742142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaef275-1dff-465b-9e3c-f885675cfc1f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.742857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaef275-1dff-465b-9e3c-f885675cfc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.744452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9cf7-dc9a-46cb-9ab6-b466c20d31b4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:44.762062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-cbaef275-1dff-465b-9e3c-f885675cfc1f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:44.763672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cbaef275-1dff-465b-9e3c-f885675cfc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:44.781468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:44.782277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.782825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.783106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.793830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-37582804-0efc-4c76-aa6b-1f105567afc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:44.801653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:44.802781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:44.803537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 31/66] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:44.804725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f8afd-48a0-43c0-a860-ae309d804d54 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.804725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f8afd-48a0-43c0-a860-ae309d804d54 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.804847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f8afd-48a0-43c0-a860-ae309d804d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.824572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5f8f8afd-48a0-43c0-a860-ae309d804d54 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:44.825710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5f8f8afd-48a0-43c0-a860-ae309d804d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:44.851516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 36/67] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:44.853021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:44.854036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e196401-e04c-481d-ad5f-0334ba17bdf7 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.854403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e196401-e04c-481d-ad5f-0334ba17bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.854688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e196401-e04c-481d-ad5f-0334ba17bdf7 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.856467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 37/68] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:44.857399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e196401-e04c-481d-ad5f-0334ba17bdf7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:44.903862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 32/69] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:44.905479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:44.905776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:44.906053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:44.924582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:44.926174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:44.945911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:44.989307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e196401-e04c-481d-ad5f-0334ba17bdf7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.001700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 33/70] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:45.012688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.034746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.035843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.036457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.036801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.037583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:45.037896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.039415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.048860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 38/71] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.049730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.118194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.127530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8657-b767-4afa-beda-2e48b95561ec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.158215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b8d2-06c4-4ae1-a6f7-00cf696aae6c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.167080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.167990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.168290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.168569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.179621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:45.187376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:45.188947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:45.189913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:45.190762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 34/72] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:45.191368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.207784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.207784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.207784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.207784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.217425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:45.225215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:45.226168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:45.226961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 39/73] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:45.227581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.308026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1c7e9-bce4-462b-ad9a-2bc3b067a4a8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.334084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c972-8563-46cd-9ad4-6446c0bd36a8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.346305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.347221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.347506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.347994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.359229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:45.367161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:45.367622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:45.368578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:45.369312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:45.369865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.369865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.369988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 35/74] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:45.370049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.370110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.384562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.386109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.392790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 40/75] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.392790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.393460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.438587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.482541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcdfd-a203-46b8-8804-9a142722c852 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.492548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295567ec-ce96-45d7-a85c-fea356268c3c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.515905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.516578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.516866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.517150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.517899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:45.518210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.519602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.527133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.527437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 36/76] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.527839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.528129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.528286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.528437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.529161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:45.529469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.530849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76861d64-74f7-4e3c-858a-43f30b5eb49f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.539445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 41/77] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.540728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc56-226f-4e4d-8f1f-a24f2812b594 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.541021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc56-226f-4e4d-8f1f-a24f2812b594 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.541297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc56-226f-4e4d-8f1f-a24f2812b594 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.543039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 42/78] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:45.543753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6bc56-226f-4e4d-8f1f-a24f2812b594 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.612061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.638062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04d89-6670-4ee0-aadc-11034f494d7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.646313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6bc56-226f-4e4d-8f1f-a24f2812b594 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.653085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.671701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.672396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.672679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.672952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.673700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:45.674004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.675397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.680746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.681397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.681690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.681973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.684099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 37/79] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.684896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.750291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.751801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8c7cb-b909-4e76-ad06-8b6cf9250f2b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.784178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b1b67-2964-41a6-923a-1509a83a6ad1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.818983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.819961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.820249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.820526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.821978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.823428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.825118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 38/80] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.825883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.834486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.850516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 43/81] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:45.851815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.852110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.852392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.863261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.869681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:45.871431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:45.925946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24c906-7e22-46bd-9cd6-b13fb1c32c3b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:45.936666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 44/82] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:45.947926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:45.959892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:45.960787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:45.961076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:45.961361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:45.962591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:45.964069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:45.966271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 39/83] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:45.967098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.037156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98107f76-6d28-4732-a30f-eecc2bb072fc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.041183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.067159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d848d9-bdea-4a24-8293-56875670fb28 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.070184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.071052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.071338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.071613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.072845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.074895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.076561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 45/84] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:46.080989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.084220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.104874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.105563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.105853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.106136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.172676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.174059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.182089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b87c2-b848-424e-bdb4-84f83fabb635 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.216858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.217722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.218008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.218305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.219475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.220946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.222678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 46/85] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:46.223442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.248306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.273200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 40/86] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.274093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.299584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.328983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5af02-5f1a-4a18-aea2-196340e1e3a4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.362513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.363301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.363585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.363892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.365058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.366516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.368403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 47/87] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:46.369242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.372891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f08479-0265-4331-b619-bc7c6fd127c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.406246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.406901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.407191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.407472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.449739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.472130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166a76d-f002-48e2-b54f-de7391e93c7a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.475067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.476586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.508053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.508706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.508993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.509276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.520172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.574630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 41/88] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.576054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.581704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.583268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.650424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.674750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedea3f-4c66-4277-a7cf-77b8d29086b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.680872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 48/89] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.681709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.709746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.710406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.710689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.710966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.711828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:46.712142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.713594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.719091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 42/90] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:46.719877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.728614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.797110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e7da-3680-4423-b766-092843b7e5c6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.836983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4c8f-9b5a-4594-8a53-976e58341385 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.837816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.838707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.839000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.839287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.840596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.842102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.843893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 49/91] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:46.844704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.882184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:46.882867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:46.883157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:46.883440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:46.884203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:46.885662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:46.898738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.913092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 43/92] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:22:46 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:46.913919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:46.984845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82d49b-0f13-4346-97ed-855241d38189 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:46.998858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:47.037086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:47.037994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:47.038848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:47.039159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:47.040509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:47.043405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:47.046145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 50/93] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:47.046995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:47.067561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624743bd-d82a-4376-a669-c7f783f1d27e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:47.113081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:47.114404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:47.114689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:47.115216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] PATH_INFO: `/v3/projects/051506490c954754b68f754bdfadbb0d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:47.116479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] RBAC: Authorizing `identity:get_project(project_id=051506490c954754b68f754bdfadbb0d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:47.118332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:47.120263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 44/94] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:46 2022] GET /identity/v3/projects/051506490c954754b68f754bdfadbb0d => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:47.121456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:47.175110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221625d-cfe5-4d30-953b-7fa1f7071405 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:47.205602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:47.224222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:47.225038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:47.225402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:47.225759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:47.226688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:47.227096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:47.228688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:47.238858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 51/95] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:47 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:47.240836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:47.296145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9664-0452-4cb1-8dab-e6246e53ebdb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:47.301138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:47.348775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:47.349832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:47.350339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:47.350838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:47.351982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=4fc861f9b4f146eca38f05771344d34d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:47.352508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:47.356357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37c29b-1762-4e80-8038-8038c815e568 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:47.363835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 45/96] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:47 2022] GET /identity/v3/role_assignments?user.id=4fc861f9b4f146eca38f05771344d34d&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:47.384668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea655e9-39f7-4182-a260-6c6c2109c5cd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:47.434850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:47.435548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:47.435865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:47.436151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:47.437023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=051506490c954754b68f754bdfadbb0d, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:47.437343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:47.438866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf21496-c402-464e-840c-25d6b392a697 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:47.445629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 52/97] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:47 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=051506490c954754b68f754bdfadbb0d&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:49.218687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffb04-714c-4df6-8e6c-92878318a0ad None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:49.220121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffb04-714c-4df6-8e6c-92878318a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:49.220928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffb04-714c-4df6-8e6c-92878318a0ad None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:49.223053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 46/98] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:49.228594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:49.229420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:49.229927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:49.252716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:49.254217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:49.349211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 53/99] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:49.395603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ffb04-714c-4df6-8e6c-92878318a0ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:49.543849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ffb04-714c-4df6-8e6c-92878318a0ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:49.578089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:49.591771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:49.592597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:49.592889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:49.593172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:49.609072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:49.616957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:49.618072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:49.618845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 47/100] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:49.625629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:49.682910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:49.772248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac907-5cfc-4651-8d6e-4ef061d2c557 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:49.818626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:49.819641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:49.820288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:49.820966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:49.821947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:49.822453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:49.824654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:49.839381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 54/101] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:49.845062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:49.993252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f1dc4-9809-494e-8a0d-d9a1edee1f11 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.036234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.037117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.037405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.037681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.038902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:50.040398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228eb52-19bb-4fae-9072-479899b47a8f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:50.042048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 48/102] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:49 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:50.048222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.148135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:50.158469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af796-18d4-4527-81dc-50c268343341 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.202516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.203731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.204238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.204709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.206186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:50.207646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:50.208221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce66ac-4848-48fc-9119-840b01f5fdf3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.208714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce66ac-4848-48fc-9119-840b01f5fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.209189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce66ac-4848-48fc-9119-840b01f5fdf3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.209394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 55/103] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:50.211059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 49/104] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:50.216837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.221520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.222393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.222871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.240813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:50.242597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:50.316967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:50.353740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 50/105] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:50.387871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc68999-c652-4dab-8d20-f6c6b3bf001f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.424201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.432591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.434091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.434382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.434660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/7c6356db0d524b36937100ddefe10ec7/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.440509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, user_id=7c6356db0d524b36937100ddefe10ec7, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:50.442206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc50b6-bff9-4176-a5c9-9bcf08d1a21d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:50.466402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 56/106] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:22:50 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/7c6356db0d524b36937100ddefe10ec7/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:50.483124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf94a-dac8-4467-9b31-2701f46e9cca None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.483449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf94a-dac8-4467-9b31-2701f46e9cca None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.483699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf94a-dac8-4467-9b31-2701f46e9cca None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.485251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 57/107] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:22:50.498002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0104-1a08-48c2-82c7-df476f184c23 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.498309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0104-1a08-48c2-82c7-df476f184c23 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.498571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c0104-1a08-48c2-82c7-df476f184c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.531996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6f4c0104-1a08-48c2-82c7-df476f184c23 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:50.533990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6f4c0104-1a08-48c2-82c7-df476f184c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:50.560472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:50.601642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce24399-7bef-4bde-a313-a951abacb7b9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.632629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 58/108] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:50.634308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7705-bd22-4be0-ad5d-7a334fe51712 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.634775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7705-bd22-4be0-ad5d-7a334fe51712 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.635228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7705-bd22-4be0-ad5d-7a334fe51712 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.637299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 59/109] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:50.642384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72a7b9-d821-4d05-a10c-6f1e856cdee5 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.642948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72a7b9-d821-4d05-a10c-6f1e856cdee5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.643437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72a7b9-d821-4d05-a10c-6f1e856cdee5 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.644949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 60/110] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:50.646253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.646754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.647250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.648652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.649483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.649828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.650110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.664368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:50.666380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:50.667924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-602bae52-7519-44d0-917a-9c7bb2ffe06e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:50.673972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:50.674880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:50.675637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:50.677582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 51/111] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:22:50.678938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.679388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.679829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.703223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:50.704448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:50.749567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 61/112] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:50.749942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.756318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:50.793574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 52/113] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:50.793763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.854440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:50.873300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c0eea-63c5-46fd-aa9e-b690dd280e6d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.907579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.908423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95766f-cc19-4ed2-9d90-978cf78f61ed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:50.908540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.908712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.908993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.910172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:50.911652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750f015-31bf-4037-b074-e3a3f5fab6ff None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:50.913322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 62/114] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:50.914517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f900ff-f981-49bd-93f0-b2b519cd14f2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.914808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f900ff-f981-49bd-93f0-b2b519cd14f2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.915092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f900ff-f981-49bd-93f0-b2b519cd14f2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.920863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 63/115] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:50.921668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f900ff-f981-49bd-93f0-b2b519cd14f2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:50.948709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:50.949726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:50.950182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:50.950634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:50.951663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:50.952226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:50.953778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:50.962435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 53/116] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:50.963589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.002970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.031633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f900ff-f981-49bd-93f0-b2b519cd14f2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.072400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.072944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa62134-ac25-4d19-86cc-05bd3ec45b3a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.073202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.073493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.074303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.085564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:51.093997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:51.094821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:51.095577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 64/117] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:51.097158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.111354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.112205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.112495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.112997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.117117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.125584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-491eff11-18bd-4f8f-9614-c408e0c8d503 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:51.133388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:51.134257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 54/118] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:51.140780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.141455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.141455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.154742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:51.156063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:51.221642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d264-2563-4ed3-bfc5-fa6a9445c4b8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.230669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 55/119] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:51.230669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.271297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.271996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.272283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.272561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.273275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.274736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.279564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.303308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 65/120] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:22:51 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:51.304176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.390607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed6f5c-968d-4b92-be6e-cbaf6b532058 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.401782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.432392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.433840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.434125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.434403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.436915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.439653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.442442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 56/121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.443422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.467700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8257-957c-4185-884f-2cbbd57cb6d9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.515213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.517189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.517189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.517189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.517189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:51.517476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.519982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.531538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 66/122] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.531538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.562243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.571234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fcd38-d2dd-43cf-af09-ca18b1e369c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.607442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.608156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.608447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.608729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.609485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:51.609799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.611175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.620216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 57/123] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.620923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.652456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29f9ba-3c5f-4995-8b4c-d5abf792aec6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.681685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.693338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.694690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.695199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.695714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.708636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:51.717962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:51.718823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:51.719580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:51.722220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 67/124] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:51.723278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.730974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0ec7-8047-4028-a87e-10534f11c8e7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.766633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.767430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.767720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.768032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.769241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.770691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.772635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 58/125] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.773210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.799206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:51.840170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323552-b66b-4a32-8feb-cfa85c7d07f6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.877549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17f16-75b8-4486-a4d5-d1e0ccd659d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:51.879525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.880760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.881252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.881744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] PATH_INFO: `/v3/users/17acceac351942239ed0b33e9d4f511f` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.897405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] RBAC: Authorizing `identity:get_user(user_id=17acceac351942239ed0b33e9d4f511f)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.899316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.901469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 68/126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/users/17acceac351942239ed0b33e9d4f511f => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.902953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.912401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:51.913288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:51.913581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:51.913865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:51.915207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:51.916576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:51.918764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 59/127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:51.919514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:51.955458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.015621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.019101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ae733-2948-4efe-ad5f-8565e1ed9e99 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.024892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6e303-814d-47f7-b817-b929de215380 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.057842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.059427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.059954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.060401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.061391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:52.062117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.062579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.064142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.064642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.064766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.065126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/7c6356db0d524b36937100ddefe10ec7/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.067915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, user_id=7c6356db0d524b36937100ddefe10ec7, role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.069157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.075803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 69/128] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:52.077917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.089046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 60/129] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:22:51 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/7c6356db0d524b36937100ddefe10ec7/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:52.090249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.202914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da27d6-af28-4f1c-8c2a-51656d91ef18 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.208265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b05035-ce56-48ea-8b91-682dfc6bd671 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.214951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.258217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.261695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.262387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.263485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.265267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.267720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.270022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 61/130] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:52.272592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.289352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.290416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.290857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.291287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.317611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.383553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.385412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.401990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aca59f-7eb4-4e06-b924-81c734ea7e64 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.447655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.448695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.449083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.449413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.450698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.452257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.453974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 62/131] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:52.454805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.499251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 70/132] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:52.500788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.509783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.546983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.560004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553f3f-b7ec-4631-aece-d15a7d2aba9f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.594651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.595647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.596037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.596391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/4fc861f9b4f146eca38f05771344d34d/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.598473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, user_id=4fc861f9b4f146eca38f05771344d34d, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.600025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.614570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 63/133] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:22:52 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/4fc861f9b4f146eca38f05771344d34d/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:52.616584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.685667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2ec3-d874-4c94-99d6-1724b894db1f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.757209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.758105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.758385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.758657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] PATH_INFO: `/v3/projects/051506490c954754b68f754bdfadbb0d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.760041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] RBAC: Authorizing `identity:get_project(project_id=051506490c954754b68f754bdfadbb0d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.761591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978759a-1a35-4d33-a375-1f890a268756 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.763432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 71/134] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/projects/051506490c954754b68f754bdfadbb0d => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:52.771155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87e1b-5d55-4265-92bd-de6e912443d5 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.771503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87e1b-5d55-4265-92bd-de6e912443d5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.771727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87e1b-5d55-4265-92bd-de6e912443d5 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.773693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 72/135] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:52.782098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87e1b-5d55-4265-92bd-de6e912443d5 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:52.828118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc0085-7ae1-4631-9b8f-abbd14a6812a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.872286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.873454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.873743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.874186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.876021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.877651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884843f-d64d-405b-a09c-ec4a71c2e2d4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.880121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 64/136] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:52.881280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.881568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.881847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.899514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.902360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:52.903370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:52.918335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.937613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87e1b-5d55-4265-92bd-de6e912443d5 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:52.975872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:52.977495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:52.977986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:52.978528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] PATH_INFO: `/v3/projects/051506490c954754b68f754bdfadbb0d/users/7c6356db0d524b36937100ddefe10ec7/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:52.981414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] RBAC: Authorizing `identity:create_grant(project_id=051506490c954754b68f754bdfadbb0d, user_id=7c6356db0d524b36937100ddefe10ec7, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:52.983212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:52.986525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 65/137] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:52.991834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.004585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 73/138] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:22:52 2022] PUT /identity/v3/projects/051506490c954754b68f754bdfadbb0d/users/7c6356db0d524b36937100ddefe10ec7/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:53.024104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.145630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd898b-c87e-40cc-a99e-0005048c0c38 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.146602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1c86-50dc-46a4-a8b1-e130c15679ab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.186371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.187640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.188193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.188748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.191157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.207283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:53.216563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:53.217543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:53.218370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:53.226451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 74/139] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:53.228171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.233164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.233869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.234161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.234442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.235819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=17acceac351942239ed0b33e9d4f511f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:53.235819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:53.237224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:53.249802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 66/140] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:52 2022] GET /identity/v3/role_assignments?user.id=17acceac351942239ed0b33e9d4f511f&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=64226166ce314ad1869e77142d398a6b => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:53.413161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28fdba-852a-4550-8000-129db49ca0f4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.474674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.475728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.476349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.476826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.477777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:53.478275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:53.479964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:53.490524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 75/141] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:53.500858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.526777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.650965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2817d-022a-425c-b83c-48e5b3a4ea7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.685342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.686149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.686437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.686717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.687961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:53.689421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f69111-e4eb-4601-978d-5c176c6cae60 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:53.691267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 67/142] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:53 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:53.696681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.771200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.828208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b4e09-e03f-4607-a48c-75ae6074639d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.875307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.876011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.876301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.876583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.946989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:53.948445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:54.005376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.017720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6114-f5c9-4d3c-851c-c784781c1450 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.018016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6114-f5c9-4d3c-851c-c784781c1450 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.018296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6114-f5c9-4d3c-851c-c784781c1450 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.019841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 68/143] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:54.024768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.025064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.025346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.043878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:54.045189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:54.080510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 76/144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:54.086040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:54.224017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 69/145] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:54.273488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:54.276322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561aaf2c-0481-4654-8fcf-abe3511c8c68 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:54.293918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.337777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:54.339283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.339774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.340253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.343690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:54.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:54.347685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 77/146] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:54 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:54.353155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:54.389513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.520054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ae421-112f-452f-835c-a2bb4d24df80 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:54.566056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4406-d365-4727-8752-a8f90ee562f8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:54.609833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:54.610655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.610948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.611230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.616477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:54.618211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:54.619891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 70/147] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:54.626752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:54.631131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:54.631813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.632115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.632373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.633212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:54.633524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:54.634968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fff2b-e574-4ce2-bcdb-3c56cdcef9bc None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:54.646450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 78/148] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:54 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:54.728100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.760210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3b10-edef-4867-8ee4-fbcb899e1a05 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:54.806478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:54.807082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:54.807373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:54.807656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:54.808436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:54.809896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde854dc-e283-44e0-b95c-0104ee4601e4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:54.836682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 71/149] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:22:54 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:22:55.140113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81b53-69bf-4855-9fea-33259d9c358f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.140635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81b53-69bf-4855-9fea-33259d9c358f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.140635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81b53-69bf-4855-9fea-33259d9c358f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.142329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 79/150] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:55.148080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.148374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.148715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.174226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:55.175420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:55.250277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 72/151] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:55.279426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81b53-69bf-4855-9fea-33259d9c358f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:55.399028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:55.426110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81b53-69bf-4855-9fea-33259d9c358f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:55.468329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:55.469148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.469481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.469761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.481199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:55.489525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:55.490348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:55.491533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 80/152] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:22:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:55.497454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:55.589521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:55.637487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c508ab-005f-4ada-ab8f-9eed1b531396 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:55.675530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:55.676586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.677060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.677522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.678524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:55.679063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:55.680676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:55.690607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 73/153] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:22:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:55.696130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:55.821170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a8eeb-defb-43cf-8ac9-e474461f917c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:55.858444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:55.859341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.859625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.859927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.861156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:55.862635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e47070-2879-4926-9072-4d22a5525b14 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:55.865390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 81/154] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:55 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:55.871401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:55.877131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad4ba-a90c-4563-99b3-940002499cd2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.877420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad4ba-a90c-4563-99b3-940002499cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.877702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ad4ba-a90c-4563-99b3-940002499cd2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.879211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 82/155] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:22:55.884619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:55.885142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:55.885636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:55.903513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:55.905053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:55.917635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:55.991073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 83/156] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:56.019023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.095621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7e03-4cab-4ef4-9254-2021a49ee360 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.135738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.157575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.158627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.159179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.159297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738fbce-dd5e-4e15-9880-ea488acbca2d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.160246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.195888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.196693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.196974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.197250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.208196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-2898eb2f-7a94-44d9-9273-0c2e0b9a5c3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:56.215962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:56.216916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:56.217702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 84/157] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:56.218825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce860041-7e7b-402b-b1a1-54183bc693c3 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.218825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce860041-7e7b-402b-b1a1-54183bc693c3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.218969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce860041-7e7b-402b-b1a1-54183bc693c3 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.220260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 85/158] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:56.221449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b87f-e13a-407c-be35-fc468c6dee15 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.221737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b87f-e13a-407c-be35-fc468c6dee15 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.222013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b87f-e13a-407c-be35-fc468c6dee15 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.223247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 86/159] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:56.228321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4856d0e-a153-4e73-9ea3-8e262b312453 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.228606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4856d0e-a153-4e73-9ea3-8e262b312453 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.228887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4856d0e-a153-4e73-9ea3-8e262b312453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.241322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.247943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b4856d0e-a153-4e73-9ea3-8e262b312453 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:56.249138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b4856d0e-a153-4e73-9ea3-8e262b312453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:56.251290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.252961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.321295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 87/160] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:56.322827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.323118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.323397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.339949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:56.341185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:56.366471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 74/161] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:56.367671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.395794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.417365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 88/162] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:56.418732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.486074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c06be-06cd-4dfa-be3d-1e342b9e62f1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.515875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.527506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.528275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.528558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.528832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.529735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:56.530048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.531556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.540890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 75/163] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:56.546967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.553077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42d13-cbca-4819-9ab3-406de4bec42d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.591555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.592393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.592676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.592953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.603701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:56.612181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:56.614295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 89/164] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:56.616236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.644452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.664162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577376-043e-431f-a7ae-d451fc738d44 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.702421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.703742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.704315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.704815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.706222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.708009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.710145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 76/165] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:56.711548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.725430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086aa8b-74f9-43bf-a4bd-24f1ad73877d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.752770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.763152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.763984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.764269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.764551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.775974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:56.784576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:56.785471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:56.786294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 90/166] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:56.786970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.838169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16260b5d-466f-4ad8-b80e-332874ebd35f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.880190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.881516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.882027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.882498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] PATH_INFO: `/v3/users/4fc861f9b4f146eca38f05771344d34d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.884279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] RBAC: Authorizing `identity:get_user(user_id=4fc861f9b4f146eca38f05771344d34d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.886651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.888842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 77/167] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/users/4fc861f9b4f146eca38f05771344d34d => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:56.889871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.901525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd77035-82ec-4682-a694-ea56c9feb37c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.917754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.937365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.937834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.938120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.938395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.939152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:56.939460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.940918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.950180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 91/168] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:56.950795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.002443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.012096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c84d-56ea-4e87-910f-bc9b0f125c87 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.050397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.051454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.051999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.052473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.053658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:57.054171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.055861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.056471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f254-bfd0-4cb8-80f9-a61ae965717f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.062737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 78/169] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.063997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.117197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.118283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.118464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.118965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.120067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:57.121042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.122396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.140727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 92/170] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.141701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.165428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.184921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d939d86-ee30-4833-93cd-017f97f5ba80 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.222968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.224050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.224538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.225014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.306730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.309421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.332948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f8a59-8ec2-4104-9012-ab7b42c78ce8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.344787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.370141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.371087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.371373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.371654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] PATH_INFO: `/v3/users/7c6356db0d524b36937100ddefe10ec7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.375157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6356db0d524b36937100ddefe10ec7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.376647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.378487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 93/171] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/users/7c6356db0d524b36937100ddefe10ec7 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.379201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.419318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.441563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 79/172] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:57.442971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.487280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991112ba-a344-4749-aca8-a4a6866b7946 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.526417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.527940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.528298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.528648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] PATH_INFO: `/v3/users/17acceac351942239ed0b33e9d4f511f` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.529976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] RBAC: Authorizing `identity:get_user(user_id=17acceac351942239ed0b33e9d4f511f)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.531529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.533282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 94/173] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/users/17acceac351942239ed0b33e9d4f511f => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.534151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.561987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2619d-cf18-4196-bd42-ce967a52eb01 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.588378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.619483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.620184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.620505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.620783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.664610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac99f9e-f54b-4611-b247-1d0f3ce73e44 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.689181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.706819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.708351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.710723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.712624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.713151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.713639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.715647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.717327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.720278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 95/174] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.721710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.822802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 80/175] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:57.824145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.852673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391225f-6a29-4762-b7d9-b4161d2861ec None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.868207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.904978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.906252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.906812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.907328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.908821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.910507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.912497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 96/176] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.913682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:57.914539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:57.931881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6104c-8d9e-4980-973a-c3421924a205 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:57.969772 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:57.970723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:57.971165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:57.971596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:57.972653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=4fc861f9b4f146eca38f05771344d34d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:57.973133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:57.974776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:57.982307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 81/177] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/role_assignments?user.id=4fc861f9b4f146eca38f05771344d34d&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:57.983333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:58.034050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc379a6-5fb4-447b-8d49-dcc0fa035aab None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:58.074306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:58.075550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.075728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.076134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] PATH_INFO: `/v3/projects/051506490c954754b68f754bdfadbb0d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.077528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=051506490c954754b68f754bdfadbb0d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:58.079258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:58.081806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 97/178] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:57 2022] GET /identity/v3/projects/051506490c954754b68f754bdfadbb0d => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:58.087496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:58.098028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9f0a3-c941-47eb-91d1-4b9fac391e18 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:58.160847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:58.162110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.162614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.163229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/17acceac351942239ed0b33e9d4f511f/roles/64226166ce314ad1869e77142d398a6b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.166601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, user_id=17acceac351942239ed0b33e9d4f511f, role_id=64226166ce314ad1869e77142d398a6b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:58.168907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693cd6f-c022-4bb2-ac1c-aa6deb927176 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:58.195507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 82/179] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:22:57 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/users/17acceac351942239ed0b33e9d4f511f/roles/64226166ce314ad1869e77142d398a6b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:22:58.218550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:58.277812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd93b-504a-4f07-b0e5-e5042a37d4c8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:58.325863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:58.326791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.327238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.327657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.328696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=051506490c954754b68f754bdfadbb0d, user.id=7c6356db0d524b36937100ddefe10ec7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:58.329153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:58.330761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f3300-931e-4609-93ef-dcbe42689570 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:58.338428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 98/180] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:22:58 2022] GET /identity/v3/role_assignments?user.id=7c6356db0d524b36937100ddefe10ec7&scope.project.id=051506490c954754b68f754bdfadbb0d&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:58.668699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94275d-fe5b-4a19-8589-ae7f07c43355 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.669180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94275d-fe5b-4a19-8589-ae7f07c43355 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.669335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94275d-fe5b-4a19-8589-ae7f07c43355 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.670640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 83/181] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:58.677731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.678015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.678290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.696823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:58.698350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:58.777250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 99/182] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:58.802665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94275d-fe5b-4a19-8589-ae7f07c43355 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:58.816889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:58.926047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94275d-fe5b-4a19-8589-ae7f07c43355 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:58.967883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:58.968707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:58.968992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:58.969268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:58.985319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a6f28046-2673-4827-9ce6-a212d5f46d8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:58.989448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:58.990638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:22:58.991421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:22:58.994129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 84/183] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:22:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:22:59.000725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.053933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.092532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39743d-6c26-409a-9d83-df91b33f9fde None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.092990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39743d-6c26-409a-9d83-df91b33f9fde None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.093383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39743d-6c26-409a-9d83-df91b33f9fde None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.094869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 85/184] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:22:59.100192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcd3d-bb55-4f91-bb52-f11a66e1048a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.100597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcd3d-bb55-4f91-bb52-f11a66e1048a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.100967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dcd3d-bb55-4f91-bb52-f11a66e1048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.113173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd93f06-820f-4908-9a11-cb90fc9cc3af None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.119813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-295dcd3d-bb55-4f91-bb52-f11a66e1048a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:59.121516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-295dcd3d-bb55-4f91-bb52-f11a66e1048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:59.152635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.153313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.153593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.153864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.154607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:59.154911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.156414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd1af1-21bd-44cd-887e-de38d5763e6d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.164770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 100/185] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:22:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:59.165955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00821f-7f19-4fbd-a8fd-8d52c37b7c45 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.166244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00821f-7f19-4fbd-a8fd-8d52c37b7c45 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.166518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00821f-7f19-4fbd-a8fd-8d52c37b7c45 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.167879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 101/186] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:59.171202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00821f-7f19-4fbd-a8fd-8d52c37b7c45 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.195571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.201124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 86/187] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:22:59.209178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.209494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.209804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.232415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:59.233704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:59.305736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00821f-7f19-4fbd-a8fd-8d52c37b7c45 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.312008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 87/188] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:22:59.312008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.320355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.361347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.362323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.362609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.362880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.378055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.379968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.386841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 102/189] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:22:59.386841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.454551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369a48a-2f8a-49cb-a439-9de412360540 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.497472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.506252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.507363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.507832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.508295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.510241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.513350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.522538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 88/190] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:59.522538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.524705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d9f45-e4c9-43e5-9311-7358151050fe None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.566299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.567149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.567432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.567714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.579447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:22:59.587324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:22:59.588398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:22:59.589236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 103/191] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:22:59.589870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.622299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.641490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28377e-3ecd-4e67-a6d4-ef7a61fe4ca5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.680903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.681569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.681846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.682118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.704843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220504f-b92f-4b9d-97bf-4f1acff893ad None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.739646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.740433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.740718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.740997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.741711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.743436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.764423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 104/192] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:22:59 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:22:59.765265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.775488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.779630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.781607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.889928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.912761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5a6f0-cebf-46a8-b37f-a5433487d166 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.917707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 89/193] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:59.924118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.953805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.954547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.954831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.955155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.956016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:22:59.956440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.957819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:59.966895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 105/194] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:22:59.972309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:00.035435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.044397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14ac16-e930-4412-8c54-7865763a01bd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.094372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.095797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.096101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.096381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.097926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:00.099544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.102015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 90/195] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:00.108720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:00.114493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ec138-e1cd-4bd7-8376-96081939723b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.158649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.159539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.159858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.160663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.161959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:00.164352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.166846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 106/196] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:00.172442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:00.186659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.227808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb736b5b-e545-46d9-a10a-7754f64e5f68 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.268354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.269027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.269313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.269589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.270435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=7d29ba33b7bf4d59b4118874d42c0064) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:00.270749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:00.272322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.279245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 91/197] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:00 2022] GET /identity/v3/role_assignments?user.id=7d29ba33b7bf4d59b4118874d42c0064&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:00.301242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8ac1-881b-45a6-960e-b9c73e338e0c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.340401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.341065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.341353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.341635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.428413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:00.429927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.442544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.543378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 107/198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:00.552935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:00.616123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.693364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c5140-6182-4a22-ba29-5e204a46fc6c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.729441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.730356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.730638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.730910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.732744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:00.734216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d26db-89e0-46fa-b2eb-8dab803e2b4d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.735971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 92/199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:00 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:00.745036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:00.815584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c047e1-ecb1-4348-b342-fc0a38ebe856 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.815801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c047e1-ecb1-4348-b342-fc0a38ebe856 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.816075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c047e1-ecb1-4348-b342-fc0a38ebe856 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.817735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 93/200] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:00.830609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.830897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.831173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.860759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:00.862927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:00.921021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c5056-f8c9-4e7e-b05c-4df259bd0629 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:00.942473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.981773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 94/201] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:00.994529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:00.996119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:00.996566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:00.997017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:00.998448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=7d29ba33b7bf4d59b4118874d42c0064) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:00.998961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:01.001485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:01.008847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:01.013179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 108/202] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:00 2022] GET /identity/v3/role_assignments?user.id=7d29ba33b7bf4d59b4118874d42c0064&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:01.039200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:01.157905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebcdf-a651-452e-9df9-71f5a1fa540e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:01.207440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:01.208468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:01.208758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:01.209033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:01.223246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:01.232236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:01.233127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:01.233898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:01.236663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 95/203] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:01.242885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:01.379196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8e704-044a-4654-9553-ef61dd2fd67b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:01.414585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:01.415590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:01.416151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:01.416656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:01.417704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:01.418214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:01.419886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:01.435783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 109/204] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:01.442320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:01.535797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:01.552027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d760e36-9c30-47c6-bcb2-0377117e5f1d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:01.596106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:01.597211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:01.597669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:01.598103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] PATH_INFO: `/v3/users/17acceac351942239ed0b33e9d4f511f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:01.599462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] RBAC: Authorizing `identity:get_user(user_id=17acceac351942239ed0b33e9d4f511f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:01.601096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:01.602871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 96/205] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:01 2022] GET /identity/v3/users/17acceac351942239ed0b33e9d4f511f => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:01.609140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:01.672937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:01.763693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce161f-de38-4a2d-a140-80051d9fdd86 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:01.813834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:01.814495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:01.814780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:01.815059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:01.889406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:01.890901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:01.925880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:01.997526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 110/206] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:02.003392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:02.195496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96aea5-c8f7-4253-89f7-173b4b20b01a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:02.240539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:02.241408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.241687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.241959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.248468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:02.249924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6b024-86a2-4892-b61d-980555d3e4d8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:02.251813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 97/207] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:02 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:02.260908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:02.404142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644cd4d-77b0-4a12-bedc-25000abecfa8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:02.453977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:02.454592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.454840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.455120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.456001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, scope.project.id=f442f593f7224d8b82b24251fe7cc62e, user.id=17acceac351942239ed0b33e9d4f511f) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:02.456320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:02.457766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167078ca-4da8-436d-ad29-3ac9f8197a1a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:02.461001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:02.465297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 111/208] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:02 2022] GET /identity/v3/role_assignments?user.id=17acceac351942239ed0b33e9d4f511f&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=64226166ce314ad1869e77142d398a6b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:02.465776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612295bf-2318-4255-b493-d70f420d82dd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.466303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612295bf-2318-4255-b493-d70f420d82dd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.466601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612295bf-2318-4255-b493-d70f420d82dd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.471787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 98/209] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:02.473804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.474102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.474381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.493327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:02.494530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:02.586372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 112/210] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:02.615259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612295bf-2318-4255-b493-d70f420d82dd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:02.656837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:02.754171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612295bf-2318-4255-b493-d70f420d82dd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:02.790163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:02.791422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.791970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.792470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.804225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3c5ad2b5-4138-4917-890f-5f8bba1e0652 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:02.812455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:02.813322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:02.814103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:02.816580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 99/211] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:02.822439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:02.892104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06f61-b749-4f2c-9618-18a4e0f8644f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.892809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06f61-b749-4f2c-9618-18a4e0f8644f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.893290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06f61-b749-4f2c-9618-18a4e0f8644f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.894973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 100/212] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:02.900244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbb88-31c9-491d-a55c-2ee5cfb13081 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.900651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbb88-31c9-491d-a55c-2ee5cfb13081 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.901032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbb88-31c9-491d-a55c-2ee5cfb13081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.920432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bbdcbb88-31c9-491d-a55c-2ee5cfb13081 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:02.921791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bbdcbb88-31c9-491d-a55c-2ee5cfb13081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:02.940656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169d3a9-2ee7-4376-b213-222a3463ab96 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:02.973613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:02.981585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:02.982903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.982903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.982903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.983531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:02.984689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:02.985242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02634b0-7dfa-47fa-a07e-d4a5ccc266ed None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:02.994758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 113/213] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:02.996124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed3d6-5c12-409a-8542-5f27601398e3 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:02.996478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed3d6-5c12-409a-8542-5f27601398e3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:02.996753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed3d6-5c12-409a-8542-5f27601398e3 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:02.998129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 114/214] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:03.002283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 101/215] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:03.002872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed3d6-5c12-409a-8542-5f27601398e3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.003545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.004057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.004291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.023444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:03.024709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:03.035300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.097334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 102/216] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:03.098199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.135505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed3d6-5c12-409a-8542-5f27601398e3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.171103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.171951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.172291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.172572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] PATH_INFO: `/v3/users/4a895ea628cf48fcb24c6722902f0fc4` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.187372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] RBAC: Authorizing `identity:get_user(user_id=4a895ea628cf48fcb24c6722902f0fc4)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.188957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.190820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 115/217] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/users/4a895ea628cf48fcb24c6722902f0fc4 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:03.191403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.209172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717637a-7378-45d4-ba1b-39f858705786 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.245733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.246544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.246828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.247100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.248305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.249843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.251404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.256420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 103/218] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:03.256420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.272297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.306506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fedc6-85a9-4b83-be97-d332d1bf5b10 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.341663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.342486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.342774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.343051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.355354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:03.363839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:03.364793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:03.365619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 116/219] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:03.368214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.375611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15df47-c714-48ca-82da-7083c4009324 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.410202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.410852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.411135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.411407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.487960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4d07-f3b3-4dc6-8c3d-763999dc5e3a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.495154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.496667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26f913-6a95-4f7a-a9e4-3e57ff4b341b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.498868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.510477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.522870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.523543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.523853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.524134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.524844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.526288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.546733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 117/220] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:03 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:03.546733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.607915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 104/221] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:03.607915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2ef4-a0cf-490b-8736-cae06d92645e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.607915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2ef4-a0cf-490b-8736-cae06d92645e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.607915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b2ef4-a0cf-490b-8736-cae06d92645e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.609041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 105/222] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:03.612625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b2ef4-a0cf-490b-8736-cae06d92645e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.676531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0af89-725a-4dda-a3bb-ed90d89143e9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.713151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.713827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.714669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.714958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.715791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:03.716096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.717579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3081b2-8b81-46bf-b366-345e8d37c080 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.726919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 118/223] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:03.728449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.728809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.729685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.747001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:03.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b2ef4-a0cf-490b-8736-cae06d92645e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.748322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:03.762580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.774859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.794063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.795378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.795658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.796478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.797800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.799473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.801421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 106/224] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:03.802104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.837377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 119/225] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:03.838507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.937744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a13a4-92a0-4e5a-98db-a1b7eacea1e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.972448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d10d-3885-4cb3-9fbc-a2f94cbffaa6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.975032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.976033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.976380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.976719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.978019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.979584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.981519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 107/226] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:03.982239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.017458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.018171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.018452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.018725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.019701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=4a895ea628cf48fcb24c6722902f0fc4) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:04.020292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.022631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.029806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 120/227] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/role_assignments?user.id=4a895ea628cf48fcb24c6722902f0fc4&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:04.031006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.048282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:04.092450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:04.162789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621da32-405e-4dc7-9a81-1b535197622c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.212042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.213511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.214031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.214610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.229295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:04.230962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00225a-0066-4391-8a2e-9804d8e3607d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:04.239688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:04.240574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:04.241354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:04.243903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 108/228] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:04.256899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.273536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.274330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.274609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.274883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.276102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.278159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.279943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 121/229] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:04 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:04.285640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.380392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:04.394657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d538f-59e6-4cd4-8566-6acfa7a8b52f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.445044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.445943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.446226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.446499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/7d29ba33b7bf4d59b4118874d42c0064/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.449407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user_id=7d29ba33b7bf4d59b4118874d42c0064, role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.450858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.466080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 122/230] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:04 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/7d29ba33b7bf4d59b4118874d42c0064/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:23:04.486311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcfcdb-89e3-4504-985e-f89aa42655c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.510023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:04.563127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.563747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.564055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.564336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.565113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:04.565424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.566839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.583152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 109/231] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:04.588795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.708198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e608a-b048-429e-a81f-2aa12f620e9a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.746398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.747280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.747601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.747904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.749124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.750576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.752399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 123/232] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:04 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:04.758004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.863346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565cbd-b86f-423c-b938-4d64284fc716 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:04.899384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:04.900289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:04.900566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:04.900895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:04.902633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:04.904112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89af6-3b45-4888-92c6-7c2b62b8c709 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:04.905881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 110/233] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:04 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:04.911682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:04.936683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:05.037813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c118b-145d-457c-9a95-84743c0b24c3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:05.074727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:05.075673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:05.076012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:05.076593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/7d29ba33b7bf4d59b4118874d42c0064/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:05.078667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user_id=7d29ba33b7bf4d59b4118874d42c0064, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:05.080138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd052469-d6db-4e65-a29c-b2c019851b01 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:05.095778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 124/234] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:04 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/7d29ba33b7bf4d59b4118874d42c0064/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:06.093931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d9e5-bcc5-446d-b2c5-c79461d7361d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.094458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d9e5-bcc5-446d-b2c5-c79461d7361d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.094458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d9e5-bcc5-446d-b2c5-c79461d7361d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.096168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 111/235] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:06.098858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314e622-51fd-471e-9a63-b3749bb8eb69 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.099279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314e622-51fd-471e-9a63-b3749bb8eb69 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.099448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314e622-51fd-471e-9a63-b3749bb8eb69 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.102444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.102773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.103061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.115588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 125/236] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:06.127430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:06.128645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:06.145780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.146068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.146345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.171007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:06.172360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:06.186610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:06.251679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 126/237] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:06.279406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.286077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 112/238] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:06.292671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:06.331071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.388233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189baa01-9f19-429a-9989-ab28ce607d79 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:06.425671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:06.426475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.426754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.427026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.438551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:06.449092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:06.449980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:06.450761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 127/239] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:06.457098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.560113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:06.585002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3784f-a656-427d-9a3e-835d8f0c6c96 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:06.623372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:06.624288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.624576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.624851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.626108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:06.628095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a6cc1-e4a2-4282-8d61-058163039cdf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:06.630756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 113/240] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:06.632458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c9c26-b4c9-4a62-bd7f-249ef9c22ed5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.632898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c9c26-b4c9-4a62-bd7f-249ef9c22ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.633176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c9c26-b4c9-4a62-bd7f-249ef9c22ed5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.635139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 114/241] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:06.640623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c9c26-b4c9-4a62-bd7f-249ef9c22ed5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.643236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a72bf2-8e25-4aa9-9bb7-db8223cf69c2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:06.680269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:06.680943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.681224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.681499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.682373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:06.682710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:06.684268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92360928-3c4f-4301-bf84-9e765c786f48 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:06.685260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:06.693617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 128/242] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:06.694937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.695226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.695503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.715905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:06.717127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:06.778980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c9c26-b4c9-4a62-bd7f-249ef9c22ed5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:06.799486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 129/243] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:06.808582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.833441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:06.834124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.834405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.834677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.835399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:06.836896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:06.861364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 115/244] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:06 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:06.862263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:06.884357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:06.932304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116605c6-292a-42f8-a4fc-60ac59ebc880 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:06.974293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:06.975186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.975463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.975736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] PATH_INFO: `/v3/users/911031eaf9e441929fd231db156a73ff` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.003533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.008906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] RBAC: Authorizing `identity:get_user(user_id=911031eaf9e441929fd231db156a73ff)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.012932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.014745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 130/245] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:06 2022] GET /identity/v3/users/911031eaf9e441929fd231db156a73ff => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:07.021020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.034779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af829c-3a75-4851-b039-685adae8bd3e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.097092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.097908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.098195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.098471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.113963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-26f1120b-b31b-40b9-a0f5-06a66302df73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:07.122385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:07.123465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:07.124568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 116/246] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:07.125372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3326-b75e-4704-b6d7-512dbec55769 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.125372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3326-b75e-4704-b6d7-512dbec55769 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.125372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3326-b75e-4704-b6d7-512dbec55769 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.126509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 117/247] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:07.132293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586748-119f-44ba-898f-06e378b8482d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.132293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586748-119f-44ba-898f-06e378b8482d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.132483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586748-119f-44ba-898f-06e378b8482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.151122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7d586748-119f-44ba-898f-06e378b8482d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:07.152931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d586748-119f-44ba-898f-06e378b8482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:07.170417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b2df-ca34-4d09-ae3f-770b76668fed None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.195990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.211917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.212581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.212871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.213145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.246913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 118/248] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:07.248211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586748-119f-44ba-898f-06e378b8482d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.314682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.317352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.355454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.380270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586748-119f-44ba-898f-06e378b8482d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.455590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 131/249] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:07.456413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.457901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.458594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.458881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.459150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.459934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:07.460239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.468524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.477320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.491190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 119/250] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:07.493811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.602563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3093c-faa2-4f78-8e2e-ccf537e2915d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.642373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.643557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.644139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.644902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.656214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:07.666115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:07.667066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:07.667909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:07.670746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 132/251] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:07.676310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.736203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc2c2f-9ced-415c-a95d-bcf9d3be9e40 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.773106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.780299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.781634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.783567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.783877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.785515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.786968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.789443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 120/252] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:07.790440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.806414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8954968-5969-48f9-86c8-4cff11ae4f7b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.811803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.844441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.845652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.846145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.846619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] PATH_INFO: `/v3/users/4a895ea628cf48fcb24c6722902f0fc4` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.848109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] RBAC: Authorizing `identity:get_user(user_id=4a895ea628cf48fcb24c6722902f0fc4)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.849798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83434d0-1962-4130-9864-fe716ace2dff None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.851727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 133/253] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/users/4a895ea628cf48fcb24c6722902f0fc4 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:07.853245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c6d6-c9f6-4c4a-9914-55983c8a37db None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.853736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c6d6-c9f6-4c4a-9914-55983c8a37db None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.854232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c6d6-c9f6-4c4a-9914-55983c8a37db None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.855683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 134/254] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:07.856706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c6d6-c9f6-4c4a-9914-55983c8a37db None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.924402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8f169-0c8c-4f54-9da0-508b3e81791c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.961240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.962304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.962850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.963323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.964421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:07.964935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.966531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.975301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 121/255] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:07.981375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.990005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c6d6-c9f6-4c4a-9914-55983c8a37db None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.033897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.034952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.035474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.035986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.037078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=911031eaf9e441929fd231db156a73ff) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:08.037597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:08.039346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcf35b-1c2e-4822-b977-b66e07212cf1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:08.046257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 135/256] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/role_assignments?user.id=911031eaf9e441929fd231db156a73ff&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=64226166ce314ad1869e77142d398a6b => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:08.047779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.048317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.048800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.073778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:08.075615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:08.092176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.097137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.148492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac16103-ece0-497d-a63d-f63937e082a0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.214801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 136/257] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:08.215506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:08.228174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.229071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.229356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.230167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.231401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:08.232896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:08.234772 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 122/258] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:08.240630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:08.365316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5755fb-00c0-4005-9ffd-a5d535cdd601 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.410306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.410306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.410306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.410306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.411549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:08.414195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:08.416409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 137/259] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:08 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:08.418074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:08.445270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.487747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db27a-dcda-44b7-a664-50d329434134 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.539885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.541869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.543029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.544283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/4a895ea628cf48fcb24c6722902f0fc4/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.548474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user_id=4a895ea628cf48fcb24c6722902f0fc4, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:08.550342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:08.571054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 123/260] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:08 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/4a895ea628cf48fcb24c6722902f0fc4/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:08.573354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:08.579788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adfbe0-3f1e-49b2-afd3-3a51328f9280 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.580121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.676629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.677553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.677843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.678230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.696141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:08.711290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:08.712334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:08.713117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 138/261] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:08.736799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:08.761783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1c37-8542-4dae-81b8-dcda7a01c13f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:08.809814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:08.810812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:08.811296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:08.811786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:08.883151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:08.885144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:08.912125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.914062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.973170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c6ac-4e9f-4a15-83b3-e036654dabf1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:09.000119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 124/262] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:09.005630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.037983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:09.040493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.040983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.041474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.044334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:09.044886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:09.046495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:09.067865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 139/263] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:09.084199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.151577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3246f9-8f3c-4350-a574-a2dadb80c28a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:09.154112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:09.190722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:09.191967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.192470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.192962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.194359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:09.196024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:09.198935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 125/264] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:09.204312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.298094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32a6f-d463-43e4-9083-d4c4b7509b43 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:09.355435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:09.368611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:09.369810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.370852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.371325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] PATH_INFO: `/v3/users/7d29ba33b7bf4d59b4118874d42c0064` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.373826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] RBAC: Authorizing `identity:get_user(user_id=7d29ba33b7bf4d59b4118874d42c0064)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:09.375722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:09.380003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 140/265] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/users/7d29ba33b7bf4d59b4118874d42c0064 => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:09.388324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.394529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17044a04-9f92-4422-b4fa-321afdc24df2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:09.466284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:09.466961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.467251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.467516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.468440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=7d29ba33b7bf4d59b4118874d42c0064) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:09.468751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:09.470407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf6b86-15e6-4958-b37c-02ef9b42cb6d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:09.485931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 126/266] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/role_assignments?user.id=7d29ba33b7bf4d59b4118874d42c0064&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:09.487524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca4ba5-fc1b-48ac-9066-0f2f9ef279b0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.487838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca4ba5-fc1b-48ac-9066-0f2f9ef279b0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.488111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca4ba5-fc1b-48ac-9066-0f2f9ef279b0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.494087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 127/267] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:09.497302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.498006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.500049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.543794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:09.544567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:09.577360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9089fdf-f052-4ba9-b74a-2b9489569353 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:09.617740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:09.618311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.618592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.618860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.629146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:09.693596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:09.699168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:09.700708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:09.734482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 128/268] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:09.833293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 141/269] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:23:09.834377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.837189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.976727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:09.982581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caebb3-a9e9-443c-85a3-2b88b5714102 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.019030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.020033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.020314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.020594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.021810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.023266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204a7c-2b2c-411c-848b-d6bee816ea89 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.025279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 142/270] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:10.026998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27288c9-72b6-4747-b6c1-37057593f7dd None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.027289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27288c9-72b6-4747-b6c1-37057593f7dd None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.027693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27288c9-72b6-4747-b6c1-37057593f7dd None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.028967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 143/271] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:10.033394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27288c9-72b6-4747-b6c1-37057593f7dd None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.062608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cec52-a1f4-4fcd-ac55-2a8c0f2b952c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.099016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.100234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.100718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.101190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.113643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-19f63a5b-461e-477f-a321-15703ea3b0ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:10.122124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:10.123254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:10.124162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:10.127385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 129/272] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:10.128973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.129460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.129930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.147401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:10.148925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:10.158529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27288c9-72b6-4747-b6c1-37057593f7dd None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.169155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:10.201962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.202626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.202907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.203226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.204195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=7d29ba33b7bf4d59b4118874d42c0064) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:10.204511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.206166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.212857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 144/273] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/role_assignments?user.id=7d29ba33b7bf4d59b4118874d42c0064&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:10.214514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.249998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 130/274] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:10.289912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:10.348173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.413673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5c5e1-0f17-41fa-b28c-7727fdc4dcb7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.455480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.456448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.456937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.457382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.458301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:10.458870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.460483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.471343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 145/275] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:10.477300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.489143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc2660-63b2-43ea-a592-39049b4b8b36 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.529428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.530176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.530461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.530736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.531938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.533417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.535364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 131/276] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:10.541637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.570990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:10.593601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:10.604579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b38bc8-d9e6-4fa4-b10e-9bb664f1a15e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.643977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.644860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.645148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.645431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] PATH_INFO: `/v3/users/d73ccc8927304856bc729884b56a0b1b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.660976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] RBAC: Authorizing `identity:get_user(user_id=d73ccc8927304856bc729884b56a0b1b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.662705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218971a-6b1d-4d8d-b8a2-2f93d1a17883 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.666800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 146/277] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/users/d73ccc8927304856bc729884b56a0b1b => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:10.666800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b47ba-7130-4fb9-9a1b-a56642f26327 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.666800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b47ba-7130-4fb9-9a1b-a56642f26327 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.666800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b47ba-7130-4fb9-9a1b-a56642f26327 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.667886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 147/278] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:10.670429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b47ba-7130-4fb9-9a1b-a56642f26327 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.678841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bfe3b-55da-4e96-bd60-23741a1f2f21 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.715144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.715851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.716135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.716407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.717118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.718557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b517bb-372c-482c-9fdb-ed23400fca51 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.741506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 132/279] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:10 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:10.743345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.743877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.744651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.773690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:10.775281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:10.790741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b47ba-7130-4fb9-9a1b-a56642f26327 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.835345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.836494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.837000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.837457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.838504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, user.id=d73ccc8927304856bc729884b56a0b1b, scope.system=all) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:10.839213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.840872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.852585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 148/280] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/role_assignments?user.id=d73ccc8927304856bc729884b56a0b1b&scope.system=all&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:10.879426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 133/281] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:10.903613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:10.923487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.036329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7caf8-15fc-49e7-8c5b-6ce44559da3a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.074741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.075423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.075707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.076034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.087056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:11.094898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:11.095820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:11.096632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 149/282] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:11.101867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.115278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.226750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a07c7-de57-4fda-9b86-cfbaa28b3823 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.264755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.265443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.265723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.265997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.266759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:11.267065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:11.268472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8171d8-76e5-40f9-bf97-7e70f8e12d2e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:11.277846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 134/283] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:11.289128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.312612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab78-6656-4727-9a5b-79596374a1f5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.312921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab78-6656-4727-9a5b-79596374a1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.313161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ab78-6656-4727-9a5b-79596374a1f5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.314877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 135/284] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:11.319536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.319846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.320122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.343069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:11.344281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:11.407976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.462481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.469790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 136/285] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:11.486499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697e1c-1447-4f22-a060-a8f42a9469b2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.492938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.561443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.562330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.562616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.562894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] PATH_INFO: `/v3/users/911031eaf9e441929fd231db156a73ff` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.564142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] RBAC: Authorizing `identity:get_user(user_id=911031eaf9e441929fd231db156a73ff)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:11.565603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:11.568031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 150/286] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/users/911031eaf9e441929fd231db156a73ff => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:11.574251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.605425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004e588-a32c-4ae6-83dd-6adadc23909a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.645285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.646526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.647035 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.647549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.660499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:11.670385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:11.671295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:11.672139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:11.675111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 137/287] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:11.683611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.751139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.785500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5952f30-6288-4ba8-8259-cdc47ea6ce8c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.822161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3833e60-1397-4eb6-9b44-4ce43ecb03ca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:11.863644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.864453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.864732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.865003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.866313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:11.870859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:11.872633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 151/288] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:11.874344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:11.875020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:11.875301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:11.875571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:11.876394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:11.876702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:11.878133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:11.880481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:11.897073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 138/289] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:11.900885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:11.903563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:12.011959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e633a1c-63fc-4e0f-b303-72bbcf4975b1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.049479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.050373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.050657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.058635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/911031eaf9e441929fd231db156a73ff/roles/64226166ce314ad1869e77142d398a6b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.058635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user_id=911031eaf9e441929fd231db156a73ff, role_id=64226166ce314ad1869e77142d398a6b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:12.058635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:12.071799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 152/290] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:11 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/users/911031eaf9e441929fd231db156a73ff/roles/64226166ce314ad1869e77142d398a6b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:12.087529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07286e-97d6-4c18-b42b-8346b8d76736 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.115788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:12.133604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.134856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.135343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.136446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] PATH_INFO: `/v3/users/4a895ea628cf48fcb24c6722902f0fc4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.137952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] RBAC: Authorizing `identity:get_user(user_id=4a895ea628cf48fcb24c6722902f0fc4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:12.139849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:12.142302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 139/291] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:11 2022] GET /identity/v3/users/4a895ea628cf48fcb24c6722902f0fc4 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:12.151618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:12.256178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:12.298410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615fee-20f5-4241-a6ac-efa7e737b3bd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.344472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.345475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.345953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.346442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.455294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:12.457136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:12.555841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:12.596485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 153/292] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:12.602200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:12.722318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed40b18-96f6-4c5c-b403-7f7119e6b241 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.760623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.761498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.761784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.762062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.763252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:12.764805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16af67-ecb5-434d-8cfd-fbabdeec2073 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:12.766575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 140/293] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:12 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:12.772656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:12.904087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026be5d-45df-48ba-8646-33cf6aecedc3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.951277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.951972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.952255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.952531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.953371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=4a895ea628cf48fcb24c6722902f0fc4) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:12.953680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:12.955105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466743a3-f332-4a08-8f6d-569b8a7696ca None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:12.961565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 154/294] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:12 2022] GET /identity/v3/role_assignments?user.id=4a895ea628cf48fcb24c6722902f0fc4&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:13.245306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04094b9-aec6-4047-a741-06df28f2b440 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.245675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04094b9-aec6-4047-a741-06df28f2b440 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.245863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04094b9-aec6-4047-a741-06df28f2b440 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.247387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 141/295] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:13.252906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e61c7-532e-476e-bf59-dac51a21d5aa None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.253195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e61c7-532e-476e-bf59-dac51a21d5aa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.253469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e61c7-532e-476e-bf59-dac51a21d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.271707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-882e61c7-532e-476e-bf59-dac51a21d5aa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:13.272906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-882e61c7-532e-476e-bf59-dac51a21d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:13.303816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.350480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 155/296] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:13.374172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04094b9-aec6-4047-a741-06df28f2b440 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.438970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.490175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04094b9-aec6-4047-a741-06df28f2b440 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:13.529617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:13.530348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.530627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.530903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.542445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:13.550758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:13.551644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:13.552437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 142/297] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:13.560052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43d2c-ab0a-4fba-926f-01760c455e43 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.560602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43d2c-ab0a-4fba-926f-01760c455e43 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.560602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43d2c-ab0a-4fba-926f-01760c455e43 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.561631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.562421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 156/298] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:13.568703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cdcb9-484c-4edc-801f-377457316faf None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.568703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cdcb9-484c-4edc-801f-377457316faf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.569027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cdcb9-484c-4edc-801f-377457316faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.587297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ea6cdcb9-484c-4edc-801f-377457316faf None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:13.588509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ea6cdcb9-484c-4edc-801f-377457316faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:13.662056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 157/299] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:13.663463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f3c6a-af69-4a13-a0de-9218b6ab2b94 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.663754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f3c6a-af69-4a13-a0de-9218b6ab2b94 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.664054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f3c6a-af69-4a13-a0de-9218b6ab2b94 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.665448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 158/300] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:13.671186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.671474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.671752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.686449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0496e0-d8e6-46e0-9f2d-6a66adcc3427 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:13.688240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:13.689416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:13.707973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.726365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:13.727030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.727305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.727588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.728345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:13.728647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:13.730561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:13.738466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 143/301] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:13.739238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.768336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 159/302] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:13.777399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.808668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.846640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6995b-0a9e-4a18-872d-0145813090fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:13.882847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:13.884003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.884462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.884745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.889211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0a6c-4ea0-49e7-bce4-ac34e1cb9c86 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:13.895737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:13.904264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:13.905118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:13.905913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 144/303] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:13.906548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.929634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:13.930921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:13.931204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:13.931617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] PATH_INFO: `/v3/users/d73ccc8927304856bc729884b56a0b1b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:13.933394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] RBAC: Authorizing `identity:get_user(user_id=d73ccc8927304856bc729884b56a0b1b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:13.934848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:13.941087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 160/304] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/users/d73ccc8927304856bc729884b56a0b1b => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:13.941087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:13.956832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.019434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ca3cc-1fc5-4f2d-8bea-5f389f173a8f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.037907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.048795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa166a4-6154-4299-a55f-013e41e0f374 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.054189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.054987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.055272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.055555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.056807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.059156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.060741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 145/305] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:14.066143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.085359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.086027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.086307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.086579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.087323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:14.087621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.089056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.101644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 161/306] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:14.101644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.175208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6834e8-7f01-4007-9612-8c543e7105fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.186346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.210979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3109f-dc5b-47f8-a831-1a2d9411108c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.214641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.215520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.215826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.216111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] PATH_INFO: `/v3/system/users/d73ccc8927304856bc729884b56a0b1b/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.217619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d73ccc8927304856bc729884b56a0b1b, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.219516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.229917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 146/307] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Jul 20 15:23:14 2022] PUT /identity/v3/system/users/d73ccc8927304856bc729884b56a0b1b/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:14.231116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.255894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.256993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.257452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.257900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] PATH_INFO: `/v3/groups` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.258788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.260807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b59719-5dd6-42e4-bf00-5bc6e82c311f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.278686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 162/308] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Jul 20 15:23:14 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:14.383540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15927f-e85a-4123-8be8-4f247830edc8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.429561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.430468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.430751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.431026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] PATH_INFO: `/v3/users/50b282168c6b40fabbfe77d60353c7b4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.451570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] RBAC: Authorizing `identity:get_user(user_id=50b282168c6b40fabbfe77d60353c7b4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.453429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41e60-f606-42ec-88a1-3a8b2c7dd610 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.453929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41e60-f606-42ec-88a1-3a8b2c7dd610 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.454153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.454249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41e60-f606-42ec-88a1-3a8b2c7dd610 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.455698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 163/309] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:14.455932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 147/310] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:14 2022] GET /identity/v3/users/50b282168c6b40fabbfe77d60353c7b4 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:14.461114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.461611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.461889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.462196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.480362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:14.481552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:14.491747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.498987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.557977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 164/311] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:14.578182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d0e-7821-4620-9df7-fd0f9fd60425 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.581735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.617789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.618831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.619310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.619836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.621406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, user.id=50b282168c6b40fabbfe77d60353c7b4, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:14.621912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.623596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.629632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 148/312] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jul 20 15:23:14 2022] GET /identity/v3/role_assignments?user.id=50b282168c6b40fabbfe77d60353c7b4&scope.system=all&role.id=64226166ce314ad1869e77142d398a6b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:14.696540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb09bd-fcea-4a18-a678-0a1efe0d3cad None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.736041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.736863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.737145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.737415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.750461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:14.758569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:14.759498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:14.760657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 165/313] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:14.765777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.872219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:14.879558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bbcd8-3f2e-4300-a7c7-5586c22f7bca None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:14.915463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:14.916279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:14.916561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:14.917375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:14.918217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:14.918526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:14.919918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:14.928855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 149/314] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:14.934710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:14.938607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:15.037547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c20712-9ff1-4134-8d75-d8d7ca5a5a80 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:15.072443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:15.073354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:15.073630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:15.073904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] PATH_INFO: `/v3/users/911031eaf9e441929fd231db156a73ff` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:15.075115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] RBAC: Authorizing `identity:get_user(user_id=911031eaf9e441929fd231db156a73ff)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:15.076648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:15.078272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 166/315] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:14 2022] GET /identity/v3/users/911031eaf9e441929fd231db156a73ff => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:15.083806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:15.189090 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072dedb-c2cc-4e7e-b3c9-475a4944ff5b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:15.222706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:15.223360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:15.223775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:15.223946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:15.249452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:15.293715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:15.295088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:15.394931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 150/316] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:15.400247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:15.482740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:15.508820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a8f52-26b1-4709-bce5-16e26129fe15 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:15.550585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:15.551363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:15.551643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:15.551992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:15.553198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:15.554636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079440d-0d23-48b0-868b-e9a6490bdec4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:15.556647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 167/317] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:15 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:15.562400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:15.633525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:15.669790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fdf66-e426-43b7-9736-96c86e332efb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:15.703704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:15.704389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:15.704668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:15.704940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:15.705766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, user.id=911031eaf9e441929fd231db156a73ff) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:15.706072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:15.707486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d3b94-af3b-4d15-b2d5-31af311252dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:15.713745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 151/318] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:15 2022] GET /identity/v3/role_assignments?user.id=911031eaf9e441929fd231db156a73ff&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=64226166ce314ad1869e77142d398a6b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:16.378058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e266c8-d742-4cf7-ba8c-d391dad610e2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.378606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e266c8-d742-4cf7-ba8c-d391dad610e2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.378606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e266c8-d742-4cf7-ba8c-d391dad610e2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.380350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 168/319] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:16.385348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.386224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.386425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.403879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:16.405068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.444431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488982e3-b1dc-4388-845f-7c068540ca1f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.444720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488982e3-b1dc-4388-845f-7c068540ca1f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.445177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488982e3-b1dc-4388-845f-7c068540ca1f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.446553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 169/320] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:16.452084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.452370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.452865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.469949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:16.471414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.482057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 152/321] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:16.505180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.541224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 170/322] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:16.556873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.562956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.610952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787a61c-f9cb-4174-a2e5-c1f5a3acb037 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.631462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.647154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.647986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.648277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.648552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.658971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:16.663996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647d968-8c55-4d3c-a6ce-79410872e5da None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:16.666755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:16.667811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:16.668643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 153/323] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:16.673664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.710343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.711144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.711425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.711700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.712899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:16.714375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:16.716170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 171/324] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:16.721374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.777955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87868a-8034-4410-8c44-9b265c95db99 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.788187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.812423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.813176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.813464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.813734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.814479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:16.814786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:16.816189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa45f41-fd45-4082-abd6-e9a7b97387ab None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:16.821114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066599d-b767-419a-afc5-361ce1cbbb5c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.824917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 154/325] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:16.824917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46628da-014d-4022-917c-020e0d4d14fd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.824917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46628da-014d-4022-917c-020e0d4d14fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.824917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46628da-014d-4022-917c-020e0d4d14fd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.824917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 155/326] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:16.830336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46628da-014d-4022-917c-020e0d4d14fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.856400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.857072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.857357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.857647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] PATH_INFO: `/v3/groups` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.858332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:16.859807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c63a6f-1b3a-4312-a9ce-ce685273ad79 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:16.868993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.875850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 172/327] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Jul 20 15:23:16 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:23:16.877047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.877504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.878248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.904839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:16.905970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.949228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46628da-014d-4022-917c-020e0d4d14fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.984614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 173/328] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:16.988075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.988977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.989264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.989540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] PATH_INFO: `/v3/users/d73ccc8927304856bc729884b56a0b1b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.990781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] RBAC: Authorizing `identity:get_user(user_id=d73ccc8927304856bc729884b56a0b1b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:16.992288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:16.994464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 156/329] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/users/d73ccc8927304856bc729884b56a0b1b => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:17.000774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:17.008523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:17.057147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:17.127918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21418a3-0cd1-418e-bbf9-f22917c848e2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:17.133282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4411-c820-469b-8262-0e2937705698 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:17.164167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:17.172652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:17.173762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:17.174047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:17.174319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:17.174735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:17.175400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:17.175688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:17.176037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:17.177034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] RBAC: Adding query filter params (role.id=b87cb4e1db1f4f789877ca5e28243e31, user.id=d73ccc8927304856bc729884b56a0b1b, scope.system=all) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:17.177442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:17.178879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:17.189747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:17.196803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 174/330] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/role_assignments?user.id=d73ccc8927304856bc729884b56a0b1b&scope.system=all&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:17.197676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:17.198679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:17.199486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 157/331] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:17.207331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:17.335259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164edfb6-c0a9-4293-b298-3e30fa044b1c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:17.375906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:17.376641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:17.377005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:17.377218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:17.377981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:17.378294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:17.379676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:17.388950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 175/332] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:17.394507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:17.529450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:17.533361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066249f5-5e8e-4090-9e04-da49eaab9d02 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:17.570172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:17.571050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:17.571333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:17.571608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] PATH_INFO: `/v3/users/50b282168c6b40fabbfe77d60353c7b4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:17.572849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] RBAC: Authorizing `identity:get_user(user_id=50b282168c6b40fabbfe77d60353c7b4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:17.574867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb672fc-2810-4c2b-9203-55efd760ccb5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:17.576548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 158/333] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:17 2022] GET /identity/v3/users/50b282168c6b40fabbfe77d60353c7b4 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:17.582196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:17.634646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:17.689038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860f70d-d52a-4b24-9fc9-8b0cd5655179 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:17.724007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:17.724877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:17.725157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:17.725429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] PATH_INFO: `/v3/system/users/50b282168c6b40fabbfe77d60353c7b4/roles/64226166ce314ad1869e77142d398a6b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:17.726932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=50b282168c6b40fabbfe77d60353c7b4, role_id=64226166ce314ad1869e77142d398a6b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:17.728349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f752976-15d0-458f-aae2-392eae91e98d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:17.738688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 176/334] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Jul 20 15:23:17 2022] PUT /identity/v3/system/users/50b282168c6b40fabbfe77d60353c7b4/roles/64226166ce314ad1869e77142d398a6b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:19.503830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbc4f2-c39f-44d3-be8a-93cfed2f863b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.504353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbc4f2-c39f-44d3-be8a-93cfed2f863b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.504430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbc4f2-c39f-44d3-be8a-93cfed2f863b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.505750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 159/335] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:19.517910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29f1d-e0ea-4c92-8f11-03e3e34c9cc7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.518812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29f1d-e0ea-4c92-8f11-03e3e34c9cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.519313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac29f1d-e0ea-4c92-8f11-03e3e34c9cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.539495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-aac29f1d-e0ea-4c92-8f11-03e3e34c9cc7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:19.541050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aac29f1d-e0ea-4c92-8f11-03e3e34c9cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:19.638293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 177/336] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:19.679994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbc4f2-c39f-44d3-be8a-93cfed2f863b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:19.694254 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e1251-c2af-4f47-bfd6-70e0b373e0ce None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.694875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e1251-c2af-4f47-bfd6-70e0b373e0ce None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.695350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e1251-c2af-4f47-bfd6-70e0b373e0ce None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.697230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 178/337] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:19.702439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.703211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.704232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.733489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:19.735030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:19.804410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:19.865041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 179/338] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:19.888425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbc4f2-c39f-44d3-be8a-93cfed2f863b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:19.902785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:19.966433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:19.967614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.968071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.968482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.000698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:20.008913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:20.009816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:20.011138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 160/339] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:20.023656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:20.038365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f415e7-92c7-4cba-9486-1a6997e8ff47 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:20.088030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:20.090117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.090603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.091051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.101159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.114185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:20.123752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:20.124840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:20.125778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:20.133288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 180/340] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:20.139289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:20.193856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.315623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb38e-084a-48f2-a189-bbd20a55beaf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:20.335918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b9f6f-3f25-468c-b021-b6b2cfb62574 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:20.400921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:20.401740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.402181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.402594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.403485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:20.403979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:20.405618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:20.408188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:20.408397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.408687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.408972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.412414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:20.412414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:20.412414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:20.419459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 161/341] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:20.431884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:20.436926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 181/342] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:20.440057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:20.631795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.642192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18c12e-0719-41d2-b507-3855d00b2692 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:20.645852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.659660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b66cd-d949-43b3-9c98-9dca7134561c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:20.693663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:20.694669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.695143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.695633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.734281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:20.735290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.735790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.736242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.779134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:20.780747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cce41-8509-4fa4-b788-ac0688e0ef99 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:20.817534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:20.819133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d25838-c7db-467b-a14a-6e8507d231a6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:20.893026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.905022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 162/343] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:20.906543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299f65-5e75-44df-b0ed-19f84561286b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.907222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299f65-5e75-44df-b0ed-19f84561286b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.907222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0299f65-5e75-44df-b0ed-19f84561286b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.908917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 163/344] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:20.910085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5cc8-f303-479a-ae5e-63bbfc0a6394 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.910465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5cc8-f303-479a-ae5e-63bbfc0a6394 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.910837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5cc8-f303-479a-ae5e-63bbfc0a6394 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.915586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 164/345] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:20.915586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5cc8-f303-479a-ae5e-63bbfc0a6394 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:20.940690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:20.950683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 182/346] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:20.952285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ce6c-4a5a-454c-889f-a8be703c8eca None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:20.952898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ce6c-4a5a-454c-889f-a8be703c8eca None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:20.953475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ce6c-4a5a-454c-889f-a8be703c8eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:20.975919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2ea2ce6c-4a5a-454c-889f-a8be703c8eca None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:20.977431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2ea2ce6c-4a5a-454c-889f-a8be703c8eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:21.035408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5cc8-f303-479a-ae5e-63bbfc0a6394 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.052222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 183/347] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:21.059792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2176d-c38f-40a1-838d-a248bbc4a6f6 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.060418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2176d-c38f-40a1-838d-a248bbc4a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.060898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2176d-c38f-40a1-838d-a248bbc4a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.079701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.080673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.080955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.081228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.082446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.085224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.085974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 165/348] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:20 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.086754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.093091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c5e2176d-c38f-40a1-838d-a248bbc4a6f6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:21.094542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c5e2176d-c38f-40a1-838d-a248bbc4a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:21.160935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.187626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 184/349] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:21.189312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a80a7-3377-4b46-a0a9-b7a6e4f1b9a3 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.189803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a80a7-3377-4b46-a0a9-b7a6e4f1b9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.190278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a80a7-3377-4b46-a0a9-b7a6e4f1b9a3 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.191736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 185/350] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:21.192789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a80a7-3377-4b46-a0a9-b7a6e4f1b9a3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.231811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ef55a-56a3-4754-bee4-7ec963a0cc0e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.270551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.276032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.278301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.278586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.278855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.281706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.282351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.285700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 166/351] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.285700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.333445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a80a7-3377-4b46-a0a9-b7a6e4f1b9a3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.377488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.378619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.378901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.379176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.392569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:21.400696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85a5e-9d78-4ee5-92c9-e10bf8803c07 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-999804c2-1036-410f-973c-57eb211cd552 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:21.400853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:21.401933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:21.402911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 186/352] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:21.403685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.403685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.403878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.423935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:21.424953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:21.440537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.450757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.451629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.451945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.452222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.459588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.461374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.467939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 167/353] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.469009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.507419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 187/354] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:21.510490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.564305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.585278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beaf70-3298-4386-a3c6-bfc9db3a74d8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.627847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.628658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.628943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.629221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.632974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7e54a-dff6-4955-82cd-23e40a40ca8e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.642529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-af8b818f-d1a4-4387-82bb-0cd293b2d965 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:21.651139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:21.652203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:21.653036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 168/355] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:21.653905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f46-b0e9-468b-844e-919fd83be436 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.653905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f46-b0e9-468b-844e-919fd83be436 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.659381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98f46-b0e9-468b-844e-919fd83be436 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.659381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 169/356] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:21.659381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98f46-b0e9-468b-844e-919fd83be436 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.672567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.674901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.674901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.674901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.675234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.677473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.683592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 188/357] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.683592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.724405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.794523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98f46-b0e9-468b-844e-919fd83be436 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.805360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69812b-a241-4e78-8f38-179d54b682ca None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:21.821202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:21.835070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.835888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.836086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.836362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.837111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:21.837419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.838827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.843530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:21.844237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:21.844519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:21.844806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:21.845635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=f442f593f7224d8b82b24251fe7cc62e) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:21.845949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:21.847398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:21.847522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 170/358] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.848288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.853919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 189/359] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:21.855281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:21.991088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfaee9-8166-41d4-97b4-117e1108088a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.001349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d0b5a-1989-48c0-976a-3213f4cad7c6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.008502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.038711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.040119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.040407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.040723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.048134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.049093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.049381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.049656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.050415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:22.050724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:22.053965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f411-0ebf-4225-ba09-911dba0a3367 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:22.061942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 190/360] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:22.062326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:22.063921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.064409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.064874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:22.071066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:22.072074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:22.073152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 171/361] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:22.074027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.089595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:22.090645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:22.131156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.175895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 191/362] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:22.180119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.210407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe6ef7-25eb-439d-ad65-e82e5f36dcc4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.262543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.263440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.263893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.264321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.265391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=f442f593f7224d8b82b24251fe7cc62e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:22.265839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:22.267421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:22.275190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 172/363] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:22.276951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.317717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61956a72-9fa4-40f8-8b27-8a3b3ed43ea8 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.328249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.359309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.360182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.360566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.360962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.408248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.444515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dc209-65ae-429d-ae7b-910499db6211 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.459939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:22.461437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:22.498614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.499287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.499577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.499939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.593973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:22.596468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:22.597936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 192/364] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:22.598846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.651591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.678295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.732160 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a41d-ae4a-4f90-8982-0dad91e5773a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.737881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 173/365] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:22.738733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.787083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.787844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.788128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.789067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.789841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:22.790152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:22.792083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:22.804364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 193/366] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:22.805214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.870631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a225b-9c4e-426a-a68e-2b96036a461f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.915642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.916666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.916958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.917260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.920644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.933567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:22.942108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:22.943243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 174/367] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:22.948630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.955645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248ed7-c99c-4476-be2e-5041a29ce5ae None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.993981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.001607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.002478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.002763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.003040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.004282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.005854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.007924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 194/368] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.008805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.091790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1180-c0fb-4b35-8405-766743b97f8c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.128806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eba70-fe5a-494a-ac07-3b6a59ed63dd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.147891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.149248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.149532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.149808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.152694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.154418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.156933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 175/369] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.158024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.177330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.178352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.178756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.179046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] PATH_INFO: `/v3/users/50b282168c6b40fabbfe77d60353c7b4` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.180466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] RBAC: Authorizing `identity:get_user(user_id=50b282168c6b40fabbfe77d60353c7b4)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.182484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.184405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 195/370] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/users/50b282168c6b40fabbfe77d60353c7b4 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.185723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.259823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.274285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.285277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132e2f6-80bf-4395-9679-0f852146e666 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.319444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebe0e-4514-42d3-af86-0b4d6e5a2d58 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.325009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.325953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.326471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.326955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.328043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:23.328567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.330246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.341012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 176/371] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.342096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.378107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.379058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.379287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.379561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.380956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.382216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.384290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 196/372] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.385107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.464061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c2b82-39ff-4933-b275-260c475b3a3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.491412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.502903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.504141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.504547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebd8c-bb4e-4899-8c73-d584aa0bf863 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.504745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.505127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.506426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.508130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.510354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 177/373] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.511984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.543721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.544513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.544695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.544966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.545844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] RBAC: Adding query filter params (role.id=64226166ce314ad1869e77142d398a6b, user.id=50b282168c6b40fabbfe77d60353c7b4, scope.system=all) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:23.546102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.547525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.549261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.564111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 197/374] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/role_assignments?user.id=50b282168c6b40fabbfe77d60353c7b4&scope.system=all&role.id=64226166ce314ad1869e77142d398a6b => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.564964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.631488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823390-e3e3-4ce4-bbb2-30aa5f7abe24 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.673306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268747c7-ae81-4046-a16b-057267422576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.674855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268747c7-ae81-4046-a16b-057267422576 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.675340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268747c7-ae81-4046-a16b-057267422576 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.675927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268747c7-ae81-4046-a16b-057267422576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.694455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3951235-5f22-4863-9a2b-9935bcdcb92d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.739635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.740619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.741121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.741501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.742818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:23.743367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.745090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.749197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.751839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 198/375] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:23.753015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.760156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268747c7-ae81-4046-a16b-057267422576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.762512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268747c7-ae81-4046-a16b-057267422576 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.821663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.875014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a55f2-67c6-4e63-8c03-5919bcbe78d4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:23.879099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 178/376] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:23.888141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268747c7-ae81-4046-a16b-057267422576 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:23.925726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:23.926587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:23.926970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:23.927416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:23.928396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:23.928805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.930429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.941660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 199/377] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:24.015515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:24.029040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268747c7-ae81-4046-a16b-057267422576 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:24.070840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:24.071724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.072133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.072408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] PATH_INFO: `/v3/projects/498a38cb72f746aabe0439ba1e952a81` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.080241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] RBAC: Authorizing `identity:get_project(project_id=498a38cb72f746aabe0439ba1e952a81)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:24.082799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:24.085148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 179/378] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:23 2022] GET /identity/v3/projects/498a38cb72f746aabe0439ba1e952a81 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:24.090996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:24.206365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c15fa5-aaab-4883-a10b-5e61e83cddf0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:24.228071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:24.243858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:24.244763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.245055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.245338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] PATH_INFO: `/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.251706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] RBAC: Authorizing `identity:get_group(group_id=602dcffd8e644e3fb775e1cf5ef9578e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:24.253439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62094ea8-a446-4100-8782-0656a2c950ef None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:24.255451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 200/379] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:24 2022] GET /identity/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:24.261322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:24.348629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75dad-1f13-44ef-a3bf-23d76b7ab917 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.348972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75dad-1f13-44ef-a3bf-23d76b7ab917 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.349195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75dad-1f13-44ef-a3bf-23d76b7ab917 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.350722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 201/380] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:24.360632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.360946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.361197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.384937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:24.386174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:24.415787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d9f9e-4340-48a8-9aaf-e288200c2c49 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:24.468884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:24.469469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.469755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.470032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.470880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] RBAC: Adding query filter params (group.id=602dcffd8e644e3fb775e1cf5ef9578e, role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=498a38cb72f746aabe0439ba1e952a81) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:24.471192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:24.472697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:24.487616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 180/381] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:24 2022] GET /identity/v3/role_assignments?group.id=602dcffd8e644e3fb775e1cf5ef9578e&scope.project.id=498a38cb72f746aabe0439ba1e952a81&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:24.517944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 202/382] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:24.553406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:24.587281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:24.681846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf8a0-5b35-4b2a-a9d1-58ba6b460303 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:24.724558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:24.725291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.725577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.725851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.736931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-451c8f09-fb8c-4a46-b157-46fa56080c52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:24.745187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:24.746099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:24.747433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 181/383] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:24.753119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:24.879849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269d2af-7433-4f4a-8a82-b96c07f62f4b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:24.898956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:24.919779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510daa39-bca2-4bbc-9672-ea30a30c4f8e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.920077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510daa39-bca2-4bbc-9672-ea30a30c4f8e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.920446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510daa39-bca2-4bbc-9672-ea30a30c4f8e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.922061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 182/384] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:24.931472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:24.932220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.932507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.932788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.933618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:24.933932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:24.935464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:24.941040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:24.941331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:24.941610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:24.956004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 203/385] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:24.962415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:24.969372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:24.971473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:25.070872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 183/386] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:25.100945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:25.124961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:25.174912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a1fb6-9c77-4c54-8b37-43786fc78e57 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.250897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.251785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.252078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.252357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.254363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:25.255824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:25.261135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 204/387] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:24 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:25.264548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:25.282448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99676c-e309-4a5a-a076-fb0e4375cb0a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.352965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.353812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.354135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.354418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.360507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:25.382039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:25.397096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:25.397993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:25.398821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 184/388] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:25.419818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:25.469047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fcd02-5dda-4c94-8eaa-4d32cb129ca2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.504721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.505589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.506067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.506506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.507954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:25.509579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:25.512035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 205/389] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:25 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:25.517659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:25.600039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:25.648425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bc7d-1dea-4d21-9b1b-1be852e64810 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.717414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.718201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.718386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.718636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.719390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:25.719693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:25.721092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:25.737429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba92c-8b38-4d03-893f-49dae65bb59d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.742896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 185/390] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:25.750108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:25.767187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:25.794600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.795489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.795797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.796070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/groups/1ad842787e844691955984244fd25b03/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.799223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, group_id=1ad842787e844691955984244fd25b03, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:25.800683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:25.823931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 206/391] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jul 20 15:23:25 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/groups/1ad842787e844691955984244fd25b03/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:25.898263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06aae7-a29d-4c42-b43e-8cc00b495be2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:25.950701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:25.951557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:25.951885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:25.952241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:25.953464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:25.954907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:25.957040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 186/392] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:25 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:25.962427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:26.079472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52224-8d7f-4796-96c8-f4c2a7306360 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:26.122663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:26.123618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.123926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.124202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.125416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:26.126865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19245fe1-f766-4437-9920-ecbf8e0f759f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:26.128624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 207/393] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:25 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:26.134740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:26.152895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:26.276586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91829c8-5e69-4ca7-9b1a-9bedab22ffd0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:26.315590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:26.316887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.317386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.317875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e/groups/1ad842787e844691955984244fd25b03/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.320361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] RBAC: Authorizing `identity:create_grant(project_id=f442f593f7224d8b82b24251fe7cc62e, group_id=1ad842787e844691955984244fd25b03, role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:26.322015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf207-7308-4fcf-a5d1-665e5c39959b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:26.337560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 187/394] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jul 20 15:23:26 2022] PUT /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e/groups/1ad842787e844691955984244fd25b03/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:26.411946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145cbf-a7be-4e2f-985c-4ccc2200a341 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.412269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145cbf-a7be-4e2f-985c-4ccc2200a341 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.412505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13145cbf-a7be-4e2f-985c-4ccc2200a341 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.414263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 208/395] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:26.419612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.420253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.420732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.445577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:26.446793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:26.515117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:26.523659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 188/396] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:26.546310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145cbf-a7be-4e2f-985c-4ccc2200a341 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:26.604675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:26.671051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13145cbf-a7be-4e2f-985c-4ccc2200a341 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:26.708735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:26.709854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.710317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.710769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.727816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-2dbc9029-02d6-4a5d-82b0-3db1e6ad42cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:26.736489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:26.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:26.738164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:26.740256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 209/397] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:26.746999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:26.776913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f75296-7aff-4ee8-8b52-48a4084ef618 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.777454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f75296-7aff-4ee8-8b52-48a4084ef618 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.777454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f75296-7aff-4ee8-8b52-48a4084ef618 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.778951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 210/398] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:26.783948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.784233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.784506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.817308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:26.818407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:26.860639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabcfed-0291-4fc0-8b2c-c934c81ad8c0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:26.899191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:26.900299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:26.900778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:26.901264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:26.902193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:26.902678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:26.904352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:26.913685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 189/399] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:26.920790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:26.989923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 211/400] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:27.024131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.045177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.090963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.094827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c6fba-cbf3-453c-925e-d9602f8b842d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.149296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.150647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.151665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.152211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.153702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.155469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270753c7-8342-489e-b84c-ee48dd1dc8b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.157706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 190/401] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:26 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:27.159361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eca73-bede-4d48-b5d3-cc972a65cd81 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.159921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eca73-bede-4d48-b5d3-cc972a65cd81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.160401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eca73-bede-4d48-b5d3-cc972a65cd81 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.166240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 191/402] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:27.167950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eca73-bede-4d48-b5d3-cc972a65cd81 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.176071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e7227-8d9b-46fb-ac13-0680394e9833 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.218918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.220108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.220614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.221097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.232360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-02d48bf1-0b26-4c09-bf87-9df998aaca8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:27.241008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:27.241919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:27.242988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:27.245816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 212/403] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:27.247445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.248000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.248485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.267497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:27.269191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:27.293793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1eca73-bede-4d48-b5d3-cc972a65cd81 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.325188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.335977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.337003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.337280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.337549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.340006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.341678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.344347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 192/404] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:27.345295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.351999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 213/405] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:27.353113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.393932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.473870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873e732-e779-4145-8208-7e96a9eb0183 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.498653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce75366-856d-4196-a98b-8306795b57ae None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.510801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.511587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.512002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.512407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.513346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:27.513765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.515211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.524214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 193/406] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:27.529839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.540759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.542254 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.542688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.543450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/groups/1ad842787e844691955984244fd25b03/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.546252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, group_id=1ad842787e844691955984244fd25b03, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.548576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.565465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 214/407] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jul 20 15:23:27 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/groups/1ad842787e844691955984244fd25b03/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:27.566802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.597407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.672982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.700437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e089ed-b382-48a5-afde-133e0636ca55 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.702899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adac7d-a80c-4b29-a648-a748a02d135d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.747144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.748653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.748874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.749288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.750688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.752764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.755452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 215/408] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:27.759057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.759881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.760744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.761029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.761941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.774945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:27.785469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:27.786345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:27.787125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:27.788234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 194/409] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:27.793939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.886077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d5c6f-2799-4455-9b6a-6c5b4f2366b7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.928903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.929827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.930154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.930388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.931576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.933671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:27.939874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 216/410] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:27.941315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.949845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d077-e7b8-4c1b-a16e-23d2539873a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.960802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.975213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:28.011754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.012465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.012752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.013030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.013791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:28.014103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.015477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.035860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 195/411] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:28.042862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:28.112638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7614b-4324-4ef4-a662-461c76ae9601 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.177952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.178763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.179048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.179328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/groups/1ad842787e844691955984244fd25b03/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.181785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7, group_id=1ad842787e844691955984244fd25b03, role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.183326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe395dbf-09fc-429f-9295-b97d9c5c56f1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.207626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 217/412] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jul 20 15:23:27 2022] PUT /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7/groups/1ad842787e844691955984244fd25b03/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:28.212210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94a71-5dcd-4cfe-b6d3-80cea54b2825 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.228371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46c92d-a5cb-4fb3-a5a1-f0a8798128aa None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.228718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46c92d-a5cb-4fb3-a5a1-f0a8798128aa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.228993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46c92d-a5cb-4fb3-a5a1-f0a8798128aa None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.230923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 218/413] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:28.236270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.236558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.237227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.266934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:28.268185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:28.295305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.296230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.296520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.296799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] PATH_INFO: `/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.299656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] RBAC: Authorizing `identity:get_group(group_id=602dcffd8e644e3fb775e1cf5ef9578e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.301163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.302892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 196/414] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:28 2022] GET /identity/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e => generated 229 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:28.310945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:28.393287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 219/415] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:28.405059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:28.426513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:28.439530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:28.464198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dcfc1-616f-47a3-88b2-a59da6c8dedc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.499384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.500428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.500724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.501002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] PATH_INFO: `/v3/projects/498a38cb72f746aabe0439ba1e952a81` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.502203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] RBAC: Authorizing `identity:get_project(project_id=498a38cb72f746aabe0439ba1e952a81)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.503799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.506436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 197/416] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:28 2022] GET /identity/v3/projects/498a38cb72f746aabe0439ba1e952a81 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:28.512168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:28.590825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36b32d-d508-4a1b-af9e-ad2e052e39a9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.627164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.628002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.628283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.628557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.652064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:28.659875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:28.660713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:28.661497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 220/417] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:28.668941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:28.766235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c7b9-b8e3-4b39-a591-422b408a8735 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.790672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:28.794587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:28.832507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.833448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.833734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.834007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] PATH_INFO: `/v3/projects/498a38cb72f746aabe0439ba1e952a81/groups/602dcffd8e644e3fb775e1cf5ef9578e/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.837614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] RBAC: Authorizing `identity:create_grant(project_id=498a38cb72f746aabe0439ba1e952a81, group_id=602dcffd8e644e3fb775e1cf5ef9578e, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.839579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.862984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 198/418] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jul 20 15:23:28 2022] PUT /identity/v3/projects/498a38cb72f746aabe0439ba1e952a81/groups/602dcffd8e644e3fb775e1cf5ef9578e/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:28.863166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ad822-f1d5-44f0-9e53-0fc2157c2927 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:28.921892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:28.922564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:28.922850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:28.923123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:28.924052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:28.924358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:28.925805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496d9b3-7407-4d5b-bef1-4a1fb7be1095 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.940860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 221/419] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:28.946384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.067594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a8b5-7bb5-4d2a-8e76-0b85dd0016af None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.086794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1357-47c7-403d-b0cd-15bd638b7a7f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.087101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1357-47c7-403d-b0cd-15bd638b7a7f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.087348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1357-47c7-403d-b0cd-15bd638b7a7f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.089576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 222/420] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:29.096516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.097131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.097506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.105780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:29.106675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.107095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.107508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.133023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:29.134229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:29.169257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:29.212493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 223/421] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:29.221501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:29.223037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:29.259935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.325224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:29.370540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7bd27-dfb9-40fc-b507-dbc9a23fac7e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.383570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 199/422] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:29.389975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.407993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:29.409032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.409319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.409595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.421249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:29.429408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:29.430283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:29.431565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 224/423] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:29.452059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.505842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:29.551692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea4e3-58b7-4298-aa82-2b6119febfdc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.565590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6002fdb-538c-4e84-b44a-c0c64de81985 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.588307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:29.603535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:29.604684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.605186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.605664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.606729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:29.607281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:29.608974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:29.620495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 225/424] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:29.625034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:29.626202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.627217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.629272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.629780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.632337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:29.636885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:29.643337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 200/425] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:29 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:29.648588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:29.803182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968937-f664-4953-a608-160c9fce6ced None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.842855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:29.843894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:29.844416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:29.844910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:29.915003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:29.917381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:29.925731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618ad71-700c-4e1b-804d-f2faf3916982 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:29.940110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.018954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.019620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.019942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.020220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.023851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 226/426] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:30.024458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.025956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.027698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 201/427] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:29 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:30.036350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.036482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.060121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.120831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.141611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762b5a-dd50-4754-8e06-38b026f71243 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.167936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dba9-9d1e-4542-a6b9-6865e21b165b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.178490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.179752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.180282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.180897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] PATH_INFO: `/v3/projects/f442f593f7224d8b82b24251fe7cc62e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.182586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] RBAC: Authorizing `identity:get_project(project_id=f442f593f7224d8b82b24251fe7cc62e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.185060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7508a-a39f-442f-9c18-a273047ae333 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.187985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 227/428] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/projects/f442f593f7224d8b82b24251fe7cc62e => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:30.189444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa8573-d0bf-4a8a-a1e3-ce5480f7562a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.189949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa8573-d0bf-4a8a-a1e3-ce5480f7562a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.190417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa8573-d0bf-4a8a-a1e3-ce5480f7562a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.192976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 228/429] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:30.194066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa8573-d0bf-4a8a-a1e3-ce5480f7562a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.210725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.211852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.212676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.213790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.214914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=f442f593f7224d8b82b24251fe7cc62e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:30.215560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.217516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61041-68b6-429f-9412-09fcd45c5561 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.226294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 202/430] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:30.228245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.228722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.230070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.272945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:30.274181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:30.336311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa8573-d0bf-4a8a-a1e3-ce5480f7562a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.407992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.408778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.409168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.409455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.410645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.412443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.419137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 229/431] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:30.424461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.431650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 203/432] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:30.451127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.456028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.459126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.554976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45829a35-1ec8-4b1b-bc19-1773f5746df3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.592126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.593120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.593636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.594126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.595199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=f442f593f7224d8b82b24251fe7cc62e) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:30.595713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.597531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.599992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316bd31-d23b-487b-adce-a61af63a90aa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.607082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 230/433] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=f442f593f7224d8b82b24251fe7cc62e&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:30.640810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.642024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.642526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.642995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.656444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-28e3ff53-8b14-4732-b994-931a9affb9a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:30.664935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:30.665846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:23:30.666716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:30.669674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 204/434] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:30.676038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.769333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.795855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ae09d-f544-4078-aef8-5d5e1023fd72 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:30.802513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804a20-2ca6-46d8-9f86-20abeb6a75f1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.803106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804a20-2ca6-46d8-9f86-20abeb6a75f1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.803572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804a20-2ca6-46d8-9f86-20abeb6a75f1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.805436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 205/435] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:30.811051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.811613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.812130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.833937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:30.835412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:30.835732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:30.836124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:30.836408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:30.836684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:30.837443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] RBAC: Adding query filter params (name=member) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:30.837753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:30.839870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:30.850148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 231/436] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:30.859216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:30.920449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.932942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 206/437] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:30.956757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.006395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3961c55-65ad-4a91-bcb8-7cbdf96fd19d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.047592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.048742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.049222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.049718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.062929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.084839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10365ab6-2113-49f5-babc-b5d02fb341ec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.137997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.139411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.139937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.140388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.144735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.146212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.156154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:31.165062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:31.165959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:31.167329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 207/438] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:31.174446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.242890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.270886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 232/439] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:23:31.276819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.295521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ec96d-7955-445f-be1b-ef300f11af18 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.325259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.331261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.331960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.332244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.332527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.333274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:31.333582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.334946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635ac70-70ad-4d8a-ad32-b01c98601bea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.343677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 208/440] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:31.344906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29227-1a87-4dbb-9179-321676b42b80 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.345190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29227-1a87-4dbb-9179-321676b42b80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.345466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad29227-1a87-4dbb-9179-321676b42b80 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.346677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 209/441] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:31.348974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad29227-1a87-4dbb-9179-321676b42b80 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.389540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a0134-89e2-4f05-9eea-299870e65a7a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.424158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.425073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.425294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.425566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.428945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.430425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255cd4a-d852-4de8-a61a-aeb22c8a109b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.432213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 233/442] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:31.437746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.438025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.438317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.454194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad29227-1a87-4dbb-9179-321676b42b80 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.455238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:31.456446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:31.470467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.488299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.488944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.489225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.489504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.522377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 234/443] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:31.523209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.559901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.561367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.563780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.629541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc4514-2866-4882-81c0-7f0f0b1b9e71 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.665715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 210/444] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:23:31.665715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.668058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.668922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.669198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.669468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.670759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.672326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.673962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 235/445] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:31.674716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.681455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.769180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77242f6-f9d6-4aaf-98fe-0a6ee5d2fb3c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.775357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1827ed-3385-487a-87bd-8a062fa9d4cb None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.784308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.801901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.802691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.802974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.803248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.809042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.809890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.810166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.810440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] PATH_INFO: `/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.811674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] RBAC: Authorizing `identity:get_project(project_id=7ec05c2e55a6470a9ff0a45fd7a2dfd7)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.813202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.813794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:31.814902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 236/446] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/projects/7ec05c2e55a6470a9ff0a45fd7a2dfd7 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:31.815660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:31.821710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:31.823374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:31.824190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 211/447] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:31.824844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.903666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.919186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87ac5-c26b-4eca-860e-c8aecbdf8cff None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.926999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de0cfc-d7d3-4dd9-b0e2-129ae95f323f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.953018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.953602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.953879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.954153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.954978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=b87cb4e1db1f4f789877ca5e28243e31, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:31.955287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.956740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.960732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.961584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.961867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.962143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] PATH_INFO: `/v3/groups/1ad842787e844691955984244fd25b03` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.962532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 237/448] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=b87cb4e1db1f4f789877ca5e28243e31 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:31.963325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.972221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] RBAC: Authorizing `identity:get_group(group_id=1ad842787e844691955984244fd25b03)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.973739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.975694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 212/449] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/groups/1ad842787e844691955984244fd25b03 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:31.982134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:32.020491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:32.090737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06f347-84e2-442a-927d-278498a8453b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.105375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ccad8-1d98-43db-bcbd-aea0c1ff5d25 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.137491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.138166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.138444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.138715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.139472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:32.139809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.141205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.148726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.149583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.149875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.150407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.154869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 238/450] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:32.155033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] RBAC: Adding query filter params (group.id=1ad842787e844691955984244fd25b03, role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:32.155033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.155033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.157644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:32.159614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 213/451] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/role_assignments?group.id=1ad842787e844691955984244fd25b03&scope.project.id=7ec05c2e55a6470a9ff0a45fd7a2dfd7&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:32.287249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d0397-5653-42a7-ad93-db396c85b455 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.289584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:32.325067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.325774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.326105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.326381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.398814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.400348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.499779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 239/452] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:23:32.505522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:32.516943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:32.604746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f9d1b-883c-4958-a0c8-da7cb298428c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.637244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.638096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.638380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.638652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] PATH_INFO: `/v3/projects/498a38cb72f746aabe0439ba1e952a81` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.639848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] RBAC: Authorizing `identity:get_project(project_id=498a38cb72f746aabe0439ba1e952a81)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.641284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.642945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 214/453] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:23:32 2022] GET /identity/v3/projects/498a38cb72f746aabe0439ba1e952a81 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:32.648684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:32.651364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:32.745344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0745-7df7-489e-a506-930ed28849e4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.778696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936553f-f07c-4876-915e-607874c53aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.779529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936553f-f07c-4876-915e-607874c53aca None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.779837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936553f-f07c-4876-915e-607874c53aca None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.780113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936553f-f07c-4876-915e-607874c53aca None admin] PATH_INFO: `/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.781260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936553f-f07c-4876-915e-607874c53aca None admin] RBAC: Authorizing `identity:get_group(group_id=602dcffd8e644e3fb775e1cf5ef9578e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.782681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936553f-f07c-4876-915e-607874c53aca None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.784288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 240/454] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jul 20 15:23:32 2022] GET /identity/v3/groups/602dcffd8e644e3fb775e1cf5ef9578e => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:32.790067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:32.866210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:32.889444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dd4cc-25b3-43b7-89d8-152556bceed1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:32.921642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:32.922287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:32.922572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:32.922847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:32.923672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] RBAC: Adding query filter params (group.id=602dcffd8e644e3fb775e1cf5ef9578e, role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=498a38cb72f746aabe0439ba1e952a81) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:32.924012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:32.925437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87f02c-fb30-471e-b773-4262da98e161 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:32.931089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 215/455] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jul 20 15:23:32 2022] GET /identity/v3/role_assignments?group.id=602dcffd8e644e3fb775e1cf5ef9578e&scope.project.id=498a38cb72f746aabe0439ba1e952a81&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:35.491390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9ae10-396a-428a-8b65-7e5000b5617d None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.491951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9ae10-396a-428a-8b65-7e5000b5617d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.491951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9ae10-396a-428a-8b65-7e5000b5617d None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.493888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 241/456] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:35.499283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d427-949d-4f21-9573-839750bda6f8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.499678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d427-949d-4f21-9573-839750bda6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.499891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d427-949d-4f21-9573-839750bda6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.524727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-95e1d427-949d-4f21-9573-839750bda6f8 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:35.526458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-95e1d427-949d-4f21-9573-839750bda6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:35.591780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6b25-9bec-45bd-892c-b510274262da None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.592064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6b25-9bec-45bd-892c-b510274262da None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.592500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6b25-9bec-45bd-892c-b510274262da None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.594263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 242/457] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:35.599669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352183-9c22-4f38-b239-06367ddc2982 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.600220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352183-9c22-4f38-b239-06367ddc2982 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.600710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352183-9c22-4f38-b239-06367ddc2982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.628432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bb352183-9c22-4f38-b239-06367ddc2982 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:35.631618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 216/458] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:35.632038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bb352183-9c22-4f38-b239-06367ddc2982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:35.649080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75aeea-86e9-4746-9c5e-684ad2303b85 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.649363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75aeea-86e9-4746-9c5e-684ad2303b85 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.649638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75aeea-86e9-4746-9c5e-684ad2303b85 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.652023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 217/459] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:35.656483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75aeea-86e9-4746-9c5e-684ad2303b85 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:35.718701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 243/460] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:35.720363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.720781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.721182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.723372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:35.740887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:35.742258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:35.749085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:35.793049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75aeea-86e9-4746-9c5e-684ad2303b85 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:35.820517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 244/461] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:35.821607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:35.841789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:35.842748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.843126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.843497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.864807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:35.866436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:35.886227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:35.887130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:35.888001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:35.888001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:35.888001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:35.888001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:35.890241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 218/462] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:35.891204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:35.935943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bbd57-135f-4609-9d92-8a215c3bf9e4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:35.973395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:35.982452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:35.983674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:35.984200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:35.984670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:35.996518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:35.998343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.006866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457c06-2296-4356-9d71-34ee2c2d7971 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:36.019505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:36.020510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:36.021500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:36.021500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:36.021500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.021500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:36.023121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 245/463] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:36.024199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.032119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.044437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.045337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.045783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.046200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.057395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.059232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:36.078871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:36.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:36.080809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:36.080809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:36.080809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.080809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:36.082824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 219/464] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:36.083788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.139195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b0a68-a12b-434c-9c84-9a3df0707114 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.175172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.176055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.176531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.177045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.177968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:36.178452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.180290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.193377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866be39-56e9-4884-a25d-f0df4bf709b8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.199119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 246/465] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:36.200238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.232222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.233214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.233656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.234069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.234936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:36.235380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.237087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.250531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.252095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 220/466] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:36.253389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.291863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.316006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bde436-b24f-48aa-97b2-c4af145011f2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.351469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.352169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.352452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.352728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.353451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:36.353783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.355154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.362196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132771c7-b799-440a-ad65-b999c6a41617 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.371820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 247/467] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:36.371820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.404791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.405466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.405754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.406065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.406779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.408247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20c53-0362-4c63-adaa-6d03ab5cd91a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.435993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 221/468] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:36.437609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ca1c-2b02-4ef3-afe3-50bc5a94b6e9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.437900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ca1c-2b02-4ef3-afe3-50bc5a94b6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.438175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93ca1c-2b02-4ef3-afe3-50bc5a94b6e9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.439469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 222/469] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:36.440395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93ca1c-2b02-4ef3-afe3-50bc5a94b6e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.501686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dd4f8-fc61-4648-8628-537f8c0b372c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.504540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.547452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.548200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.548489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.548765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.549492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.550954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ee693-9317-4380-a9d3-f2a6ea0aeea0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.580885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 248/470] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:36.583582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c64de-674c-48f3-ab59-2887c20b4227 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.584119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c64de-674c-48f3-ab59-2887c20b4227 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.584930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c64de-674c-48f3-ab59-2887c20b4227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.599832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.608989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-083c64de-674c-48f3-ab59-2887c20b4227 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:36.610507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-083c64de-674c-48f3-ab59-2887c20b4227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:36.620419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93ca1c-2b02-4ef3-afe3-50bc5a94b6e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.671331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.672127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.672413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.673275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.674023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.676071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7800166-34e9-469d-8e87-d701d5a7b99a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.703872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 249/471] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:36.705603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb099d0-fb33-4d2f-b6ed-7bce4f108bc1 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.706089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb099d0-fb33-4d2f-b6ed-7bce4f108bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.706541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb099d0-fb33-4d2f-b6ed-7bce4f108bc1 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.706806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 223/472] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/users => generated 248 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:36.708641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 250/473] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:36.731984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.732430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.732809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.733834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb099d0-fb33-4d2f-b6ed-7bce4f108bc1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.758078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:36.759278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:36.863947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 224/474] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:36.883318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.889827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.908998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb099d0-fb33-4d2f-b6ed-7bce4f108bc1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.949751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.950561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.950838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.951108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.967059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.968632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.985301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:36.988722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:36.989596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:36.990424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:36.990424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:36.990424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:36.990424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:36.991807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 251/475] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:36.997624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:37.016727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015eafc1-b9e3-4cba-917c-d4819c669fdd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:37.053606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:37.054288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:37.054582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:37.054862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:37.055626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:37.057585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:37.065991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.sql.core [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:23:37.065991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 15:23:37.065991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [parameters: {'id': 'dfc5d9c1674b475c86bd0ee9552879fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 15:23:37.065991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80207) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 20 15:23:37.084256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:23:37.086311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application yield resource Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise self.value Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application session.commit() Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:23:37.088551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:23:37.090824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:23:37.092447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:23:37.093459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [parameters: {'id': 'dfc5d9c1674b475c86bd0ee9552879fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:37.094430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 15:23:37.095430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.100790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 225/476] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:37.100790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:37.140347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201e352-a762-4e73-bc7d-e6a8321b0f74 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:37.193766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:37.194804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:37.195386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:37.195972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:37.196988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:37.197529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:37.199611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:37.217861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:37.226686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 252/477] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:37.228487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d027-9ba4-4c6a-b662-1b0418727f8e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:37.232988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:37.283030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:37.283814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:37.284062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:37.285093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:37.310601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:37.320744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:37.327623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:37.328560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:23:37.329358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:37.330830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 226/478] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:37.335995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:37.413614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39233f8-4a29-4d78-a7e7-4e3ad4b0fac9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:37.480002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:37.480678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:37.480958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:37.481236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:37.481955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:37.483455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c38b3b-d575-47d8-aa2a-7afea92b2c7f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:37.519095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 253/479] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:37 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:23:37.524572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0867f-1792-4cfb-89ae-2f578e161706 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:37.588564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:37.589261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:37.589548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:37.589830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:37.590675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:37.591012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:37.592419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee7ee8-8c5a-4e39-890b-6854a9978374 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:37.610159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 227/480] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:39.094940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd915f-1a8c-4312-bc3d-43c36709515d None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.095455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd915f-1a8c-4312-bc3d-43c36709515d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.095530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd915f-1a8c-4312-bc3d-43c36709515d None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.097125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 254/481] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:39.102401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.102783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.102967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.129010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:39.130275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:39.175214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6929-f489-46c5-ad00-bff6dd5c5759 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.175516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6929-f489-46c5-ad00-bff6dd5c5759 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.175791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6929-f489-46c5-ad00-bff6dd5c5759 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.177156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 255/482] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:39.184166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26cb-e255-49d0-a568-e801add23fb1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.184458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26cb-e255-49d0-a568-e801add23fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.184741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d26cb-e255-49d0-a568-e801add23fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.207327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.226144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 228/483] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:39.231041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-815d26cb-e255-49d0-a568-e801add23fb1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:39.232282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-815d26cb-e255-49d0-a568-e801add23fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:39.250344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.332827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.386812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe61e4-cbf0-438e-ba42-b8d00e6f7eb8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:39.409513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 256/484] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:39.419542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a98fd-84ce-4f61-a125-7f2ac4a84d2b None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.419856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a98fd-84ce-4f61-a125-7f2ac4a84d2b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.420141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a98fd-84ce-4f61-a125-7f2ac4a84d2b None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.422076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 257/485] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:39.423749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:39.424578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.424861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.425138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.432498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.432786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.433064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.446174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:39.454858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:39.455750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.456742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:39.457503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:39.457600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 229/486] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:39.457600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.459751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:39.518426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.550494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 258/487] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:39.551576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.585521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c88c4-ff5a-4560-b7b6-6617137a9f60 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:39.634040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:39.635460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.635746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.636046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.650366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:39.661723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:39.662634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:39.663946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 230/488] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:39.665200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.670063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9fbb-209a-451c-9d33-320f52538434 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:39.688013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.709244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:39.710285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.710773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.711275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.712261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:39.712773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:39.714910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:39.724247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 259/489] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:39.725426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.810879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b28093-b690-4b2f-b5a7-673b8242bb48 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:39.827751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.845637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db0222-3e70-4e58-8588-70a61691db71 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:39.852953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:39.854072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.854561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.855036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.866190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:39.875203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:39.876124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:39.876938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:39.883949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 231/490] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:39.883949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.890150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:39.891099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:39.891607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:39.892129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:39.893095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:39.893709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:39.895300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:39.905336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 260/491] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:39.906500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:39.962707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.010827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95503b27-18f5-402f-bae2-a6f326f29822 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.040300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fa740-a5a5-4a17-b85e-4218fffb386b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.048013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.048815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.049098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.049368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.061140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.062598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.076210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.076885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.077172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.077453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.078210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:40.078518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.079907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfec482-6818-4be4-bdf3-333317bcfc92 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:40.081270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.082092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.082931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:40.082931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:40.082931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.082931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:40.082931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 232/492] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.084015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.093853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 261/493] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:40.095192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e38e-661c-4004-931b-b718ead09822 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.095718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e38e-661c-4004-931b-b718ead09822 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.095854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e38e-661c-4004-931b-b718ead09822 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.097137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 262/494] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:40.100536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e38e-661c-4004-931b-b718ead09822 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.116359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.196831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22225c-4df4-4e78-93fe-67d43c5d1afe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.214518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e38e-661c-4004-931b-b718ead09822 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.216185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.232865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.233661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.233946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.234223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.245790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.247220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.252788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.253449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.253736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.254013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.254738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:40.255048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.256461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac672860-eaea-4578-a2ce-b27c677dc928 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:40.264444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.265249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.266088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:40.266088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:40.266088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.266088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:40.266088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 233/495] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.266390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.272388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 263/496] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:40.273611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae396-1bb0-467f-9417-d641a10154bc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.273906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae396-1bb0-467f-9417-d641a10154bc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.274186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae396-1bb0-467f-9417-d641a10154bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.291800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bd5ae396-1bb0-467f-9417-d641a10154bc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:40.293016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bd5ae396-1bb0-467f-9417-d641a10154bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:40.348732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.365057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 264/497] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:40.366438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa343-07e8-4914-9b5d-aea525e7dc70 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.366728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa343-07e8-4914-9b5d-aea525e7dc70 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.367004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa343-07e8-4914-9b5d-aea525e7dc70 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.368526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 265/498] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:40.369208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa343-07e8-4914-9b5d-aea525e7dc70 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.385779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffb5e2-2871-49c6-86b8-58590bc84d49 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.420125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.420928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.421215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.421492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.432840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.434281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:40.451174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.452191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.453101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:40.453101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:40.453101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.453101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:40.453101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 234/499] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.453446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.471051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.481848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa343-07e8-4914-9b5d-aea525e7dc70 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.515387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.516076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.516365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.516646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.517367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:40.517687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.519068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b5099-c66c-44f2-97a2-2bd389c97734 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.530380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 266/500] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:40.531563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.531884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.532170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.548595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:40.549866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:40.552524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82981146-0696-4e88-8d8c-4963666ccb37 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.578244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.587617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.588442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.588726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.589003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.597581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.599021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:40.614455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.615370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 235/501] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.621835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.626287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 267/502] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:40.627127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.712807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.730869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b294-0ef1-44ea-b90d-a708589a99d7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.732158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95134ec-b91c-4cc3-aa78-4acd04093223 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.763841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.764664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.764944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.765214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.766241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.767007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.767180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.767454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.768212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:40.768515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.769889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.776582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.778061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.781459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 268/503] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:40.782247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:40.797520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.798431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.799280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:40.799280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:40.799280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:40.799280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:40.799280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 236/504] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.799538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.841155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.881108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0c911-e745-445a-a069-b2331afdaf78 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.899738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18402-5ca7-4642-8ac5-b529263b83b2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:40.913795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.914593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.914871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.915140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.922908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.924909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.933924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:40.934587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:40.934870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:40.935144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:40.936017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:40.936326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:40.937726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:40.939678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:40.940736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:40.941757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 269/505] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:40.942317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:40.944970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:40.977377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 237/506] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:40.978276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.043318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421631d7-75d1-4621-9878-a5698b6fd201 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.075734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.079603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c92ef-950d-4369-ba1a-46db1504353f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.081483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.082280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.082566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.082841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.095168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:41.104912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:41.106212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:41.107030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 270/507] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:41.107738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.119107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.120008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.120289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.120559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.128687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.130134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:41.145369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:41.146251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:41.147128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 238/508] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:41.147692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.172996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.212627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04896c-31b0-43be-a16e-4b7e2eca5cdf None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.245965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.246571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.246857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.247119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5533a-c77b-40f5-955c-1f22aeef3550 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.247214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.247877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:41.248189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.249583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.261748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 271/509] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:41.262536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.281318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.281987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.282266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.282544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.283384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:41.283695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.285153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.302456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 239/510] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:41.303272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.320303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.362730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433e00-d0cf-4058-b3ec-3931397061d9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.387232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.398748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.399393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.399674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.400006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.404970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cfd63-f77d-40e3-a7d4-d5216a1a5374 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.442896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.443564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.443872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.444145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.444885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:41.445186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.446540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.454896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 240/511] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:41.455663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.470486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.471983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.536000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.558566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c367-f541-4641-b7d9-05b97c3e18f4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.570478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 272/512] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:41.571292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.592015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.592691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.592977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.593254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.594099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:41.594409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.595893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.596108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.613032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 241/513] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:41.613906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.677233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461971-3fd0-470e-8e9e-b3eb1b16b66e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.716352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.716767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52145-234c-43bd-bb12-cf347584d128 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.717026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.717306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.717579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.718281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.719731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.743309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 273/514] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:23:41 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:23:41.744164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.763565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.764293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.764580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.764859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.769278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.845255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:41.848257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.849935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.872465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec195c-50db-4b6c-8048-b905a884a47a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:41.914875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:41.916038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.916379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.916808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.930163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:41.931651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:41.953612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:41.954515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:41.955339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:41.955339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:41.955339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:41.955339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:41.955339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 274/515] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:41.955605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:41.968270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 242/516] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:41.968270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.004465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.071050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0b35b-a1f5-4746-9f44-620742a7a59e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.083318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d21de-cea6-4cf8-82f6-2376aea8e47c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.107926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.114939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.115733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.116057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.116336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.121475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.122133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.122420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.122697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.128573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.130026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:42.151892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:42.152778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:42.153592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:42.153592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:42.153592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.153592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:42.159914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 275/517] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:42.159914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.202249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.203717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.239298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.264467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5dd13-7ea2-4d70-9539-d8a556fe06fa None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.301547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.302209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.302498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.302777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.303516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:42.303870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.305268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.311660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 243/518] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:42.312753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.322052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 276/519] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:42.323478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.367137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.420050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1435d-1614-40d4-849d-bfb1fa8004de None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.433693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12196d-9c35-41d5-9235-a2c4547f3fa2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.458794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.459681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.460073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.460356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.463382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.470202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.470653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.470883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.471167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.471921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:42.472231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.473164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.473616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.474689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.490152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 277/520] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:42.490152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:42.493775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:42.495287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:42.496160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:42.496160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:42.496160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.496160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:42.496713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 244/521] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:42.497488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.599107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d8b0-3390-46b5-9769-27faaf6210f7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.600765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.601349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be3bd0-18f3-4240-b70a-e7111b1894eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.639547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.640394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.640749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.641119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.642544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.643405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.643791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.644131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.651463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.652389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.653097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.653968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:42.668217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:42.669269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:42.670354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:42.671454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 245/522] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:42.672081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:42.676423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:42.677397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:42.678307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:42.678307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:42.678307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:42.678307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:42.678307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 278/523] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:42.678708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.727899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.785207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc26-b7e5-4181-a5f2-5799e308c1bc None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.789917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2d20-0a98-44db-afe1-2f0e52781fa7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.818748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.819559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.819972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.820360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.821095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:42.821413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.823337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.826596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.827379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.827658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.827952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.834758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.840700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 279/524] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:42.840832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:42.840947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:42.855639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:42.856555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:42.857414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:42.857924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.861959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 246/525] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:42.862814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:42.953821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a374a3-71f8-4316-9aad-e60406a252e6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.975298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f49da-760a-4f98-980a-0e8e749fb833 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:42.985086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:42.989666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:42.990337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:42.990617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:42.991185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:42.991725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:42.992059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:42.993749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.012844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.013508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.013795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.014072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.014795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.015106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.016492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.034849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 247/526] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.035029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 280/527] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jul 20 15:23:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.035029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.035745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.108703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.144704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf62ab5-f6f3-4304-992a-af6dadc768bc None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.148536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0c69e-61a8-4eb5-887d-e2e312fa3628 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.180430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.181364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.181664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.181928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.182716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.183023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.184501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.185391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.186299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.186586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.186867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.193786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.195318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.206861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 281/528] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.206861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:43.209522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:43.210598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 248/529] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:43.211465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.242320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.319609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de33933-a38a-4db4-bcc4-a1a89d94a60d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.325152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e156a0-8994-43be-8856-8e2155208375 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.356283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.356944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.357222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.357453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.357567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.363168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.363990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.364274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.364549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.372371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.373823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:43.388003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:43.388865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:43.389714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 282/530] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:43.390181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.434706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.436277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.465806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.508557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f8740-9015-4eac-a60f-311067c1072c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.545861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.547804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.547804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.547804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.548064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.554161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.554161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.563014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 249/531] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:43.564009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.564934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 283/532] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.565912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.644699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.679544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6a6fb-32f6-40e9-a371-0e5fd9643af0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.691155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8c1d-f695-47bd-b9f4-1d99c3d092fc None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.711513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.721698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.722294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.722585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.722866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.723667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.724024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.725470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.735267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.736875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.736875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.736875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.737685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.738093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.739634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.745240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 250/533] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.747874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.757099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 284/534] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.758491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.859068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388de6b-ca7f-42c9-bcc2-9abf258ff1ed None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.880054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b40499-eae1-4313-af3a-d886a60bd5e2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:43.897185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:43.897341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.898567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.898800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.899173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.909947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.911393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.917154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:43.917927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.918310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.918681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.920079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=1a2388c351814d5193403a4b0d00b0af) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:43.920516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:43.922026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:43.932174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 285/535] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/role_assignments?user.id=1a2388c351814d5193403a4b0d00b0af&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:43.932174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7366aaae-9339-4c69-8da3-cee554e92f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:43.932444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:43.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 251/536] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:43.934445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582544-0c01-4992-b3be-b9085abf8939 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:43.935086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582544-0c01-4992-b3be-b9085abf8939 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:43.939442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582544-0c01-4992-b3be-b9085abf8939 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:43.939934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 252/537] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:43.942128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5582544-0c01-4992-b3be-b9085abf8939 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.004248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.093180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5582544-0c01-4992-b3be-b9085abf8939 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.095316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb1cae-2348-4559-8000-faabfe3ae513 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.146474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.147476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.147817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.148283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.148283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.149482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.149482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.149667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:44.150034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.150120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:44.150201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=51875df6e7b1444cb31e793aa8727363) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:44.155806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:44.155940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:44.156037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24780e-d4dc-444a-933d-5263ff9b4d83 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:44.159787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 286/538] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/role_assignments?user.id=51875df6e7b1444cb31e793aa8727363&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:44.161067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.161340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.161664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.171988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 253/539] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:44.183000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.195787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:44.195787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:44.309896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.336047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.347974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976f30f-fc44-41c5-8337-058cee4a5680 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.372910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 287/540] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:44.396930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.397846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.398134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.400196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.405901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:44.407435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:44.422553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:44.423467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:44.424342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:44.435793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 254/541] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:44.436647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.451427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.671428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2a228-1513-40f0-b40c-7da4dbba4f09 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.686096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843573-b2ad-438c-9943-9f01b3ab5781 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.742107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.743056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.743341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.743619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.744646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:44.744961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:44.746877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:44.764073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 288/542] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:44.767559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.792056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.793265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.793757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.794232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.804339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.826854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:44.835014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:44.835904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:44.836703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:44.847797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 255/543] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:44.858751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:44.892192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0f9a-83e5-424c-84c3-3e6e62f81575 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:44.919840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.931264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:44.931969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:44.932316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:44.932632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:44.933579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=29a153f06f9e4b459a413f2f936c1f9f) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:44.933982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:44.935409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:44.941789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 289/544] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:44 2022] GET /identity/v3/role_assignments?user.id=29a153f06f9e4b459a413f2f936c1f9f&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:45.047671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a8fa-0b98-472e-9bbb-5def0492f641 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:45.098413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:45.099104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:45.099385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:45.099658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:45.100463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:45.100773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:45.102204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:45.117310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 256/545] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:45.124024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:45.270079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:45.276734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd89b8-1832-44d3-aac2-3c647cd4bcfb None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:45.336691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:45.337499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:45.337782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:45.338122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:45.360016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:45.361571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:45.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:45.386398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:45.387241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:45.387241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:45.387241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:45.387241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:45.391889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 290/546] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:45.400616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:45.550628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:45.576294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5d7d2-05ba-4160-8342-64a275653f32 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:45.646160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:45.646846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:45.647221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:45.647498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:45.648326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:45.648636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:45.650075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:45.678094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 257/547] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:45.684157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:45.974978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa16c-1dc4-4b52-834b-550dad4c51dd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.026242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.027045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.027328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.027605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.040804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:46.042826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-2943d5b5-b39e-4778-9be1-1e370ea37dc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:46.060016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:46.061010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:46.062385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:46.063161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 291/548] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:46.067256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.130372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944241-7175-4d84-832a-b1f29b073e88 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.130805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944241-7175-4d84-832a-b1f29b073e88 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.131092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944241-7175-4d84-832a-b1f29b073e88 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.132788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 292/549] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:46.138426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.138718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.139272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.160856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:46.162468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:46.166916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.247888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb954f4-c996-4bc0-be17-e08d70261a1d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 293/550] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:46.296660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.297330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.297609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.297880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.298723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:46.299039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:46.300478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:46.314539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.318984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 258/551] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:46.324015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.327929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.475078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcaf42-c39f-48fc-82eb-a47625f34655 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.497874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d8b77-9c37-48e9-9dbb-c0e29f602fd9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.531520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.532735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.533256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.533938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.542657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.543605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.544120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.544586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.545658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.551345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-39fe5207-9fad-4930-b30b-88634f41527c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:46.561866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:46.563017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:46.564103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:46.565652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 294/552] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:46.566778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8332-a878-415c-a316-125d17e1e44f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.567061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8332-a878-415c-a316-125d17e1e44f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.567336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8332-a878-415c-a316-125d17e1e44f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.573351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 295/553] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:46.573351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db8332-a878-415c-a316-125d17e1e44f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.627457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.630639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:46.632634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc810e-61a5-4a5c-b2ea-0e1fd5d4757b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:46.711742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db8332-a878-415c-a316-125d17e1e44f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.758972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.759642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.760523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.760801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.761553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:46.761860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:46.763325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:46.763596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 259/554] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:46.765201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.765704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.766172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.773801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 296/555] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:46.774696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.785719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:46.787449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:46.861078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.862199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 260/556] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:46.863646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:46.911457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.927541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b6f2-6682-4992-bfe0-3165a816e543 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:46.971479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:46.972586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.972872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.973208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.988006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:46.989647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:46.990857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b332c-28b3-48b0-a6c1-6308aa4639e3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:47.007999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.008900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.009782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:47.009782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:47.009782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.009782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:47.009782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 297/557] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.010058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.031617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.032539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.032820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.033092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.044034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.045421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-81f9a107-e674-433f-a021-41f74c5524cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:47.064878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.066300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.067178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:47.067178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:47.067178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.067178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:47.067178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 261/558] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.067895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44bd4-b3dc-429a-9a2d-6deb8eba3057 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.067895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44bd4-b3dc-429a-9a2d-6deb8eba3057 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.068060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44bd4-b3dc-429a-9a2d-6deb8eba3057 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.074151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 262/559] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:47.075447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44bd4-b3dc-429a-9a2d-6deb8eba3057 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.134958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d05dc-5e04-46b7-9052-81c2171386bd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.151037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.174610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.175418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.175692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.176001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.178215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.188238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:47.196048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.196953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:47.197912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 298/560] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.198609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.205580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44bd4-b3dc-429a-9a2d-6deb8eba3057 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.242957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.243753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.244073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.244930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.245773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:47.246080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.247457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28803a6-4427-4822-957f-d61f552875fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.264967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 263/561] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:47.264967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.264967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.264967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.279278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:47.280491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:47.309694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68f86-63a0-40f6-bd54-c088141417e4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.344910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.345586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.345871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.346154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.346887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:47.347198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.348604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.352806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 264/562] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:47.354381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.371477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 299/563] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:47.372656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.407410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.441783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.472655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ae18a-16ce-4417-9acd-668bfb4afd69 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.476030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe009e4-e284-4c02-a57e-01a31729eb1d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.506294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.506961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.507246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.507526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.508303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:47.508620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.509143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.509950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.510051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.510231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.510501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.517139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.517960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 265/564] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:47.518563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.518738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-68407691-436a-4f2d-9ac9-829007355dba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:47.536861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.537818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:47.538877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 300/565] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.539528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0394f-663e-4fbe-a15d-1f8c423e48e0 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.539801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0394f-663e-4fbe-a15d-1f8c423e48e0 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.540070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0394f-663e-4fbe-a15d-1f8c423e48e0 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.542025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 301/566] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:47.542547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0394f-663e-4fbe-a15d-1f8c423e48e0 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.626216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b445f-e0e2-48fd-ac73-5612bb475151 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.636120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.644141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0394f-663e-4fbe-a15d-1f8c423e48e0 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.659250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.660081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.660363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.660636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.662824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.669827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.671274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.677034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.677821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.678099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.678370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:47.686284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.687255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:47.688208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 266/567] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.689016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.693037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:47.701414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.702477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:47.703298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 302/568] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.704022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.794116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6491d4f-a426-402c-814f-26090d09bb65 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.801237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367649-ad74-46c0-a8a4-c168aa1f2b45 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.826642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.827424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.827702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.828005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.834552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.835199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.835485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.835826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.836582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:47.836893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.838205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:47.838329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9b3a3-4a7d-4740-afd1-580cb96289d3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.839639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:47.850262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 303/569] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:47.852605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.852901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.853181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:47.856932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:47.857969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:47.858907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:47.858907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:47.858907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:47.858907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:47.858907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 267/570] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:47.859188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.874281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:47.875474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:47.885049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.905199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.945571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 304/571] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:47.946411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:47.963675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed305bb-d62c-4c80-aa4d-788198aa0cd9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:47.999063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:47.999739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.000053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.000330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.001178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.001490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.002919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.019875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 268/572] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.020833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.046276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befb602-434d-497b-97f4-57d5815bac6c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.081540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.082196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.082474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.082745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.083644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.083981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.085511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.094353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 305/573] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.095155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.105449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.121586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f52cc-2701-4ce2-a064-83b2c1ccc0fb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.123910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.156262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.156844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.157125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.157398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.158225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=6bd7c157dada45049506a091a97079bf) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.158533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.159967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.165229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 269/574] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/role_assignments?user.id=6bd7c157dada45049506a091a97079bf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.166035 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.205197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa931b-dc70-4752-95d2-40937ce52b27 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.247171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.247888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.248171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.248444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.249196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.249505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.250914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.264594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 306/575] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.265843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.284164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ece8a-1115-4537-829f-f69c3aa320a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.324771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.325854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.326135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.326405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.342454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:48.342700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:48.350896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:48.351842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:48.352668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 270/576] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:48.353331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.383156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.386509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e4b71-d3e1-4c7e-9da7-3a5e1e93ea44 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.423686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.424717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.425159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.425558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.437938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.440081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:48.458091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:48.458997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 307/577] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:48.460594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.466917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168fa31-d0cf-474c-8763-3b9ef2c1a90b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.504237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.505462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.505755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.506034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.517852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.519790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:48.537799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:48.538714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:48.539599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:48.539599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:48.539599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:48.539599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:48.543521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 271/578] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:48.544316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.570619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df729-3695-4948-8026-dac80163ba39 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.589873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.608681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.609477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.609754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.610026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.617806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.619281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.634342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:48.635797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:48.636748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:48.638192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:48.641524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 308/579] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:48.642356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.660299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fcdb8-ad06-41d0-8ad8-ae6939c266af None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.703823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.704827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.705253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.705535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.706279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.706693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.708089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.716748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 272/580] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.717537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.754829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784b294-ec8b-4c87-b92f-d8f7ec76a1d7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.794454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.795120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.795404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.795734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.796472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.796809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.798139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.811537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 309/581] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.812460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.833516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75319-a7b7-4897-bdf2-0eaeb15926f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.853933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.874736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.875519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.875827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.876137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.876820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.877129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.878810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.897246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 273/582] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.897246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:48.899885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:48.923279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a5aa-da55-49fd-ad45-a2fc61ded8c7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:48.957412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:48.958003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:48.958280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:48.958551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:48.959382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:48.959688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:48.961123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:48.979412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 310/583] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:48.980991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.007357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cc759-eb20-4ad5-8aa1-49ced0f9684a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.042970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.043816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.044096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.044518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.058728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.060200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:49.077916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:49.078778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:49.079657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:49.079657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:49.079657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:49.079657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:49.079657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 274/584] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:49.079956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.102299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.109789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a331-feeb-4b95-b4d4-c715a5d8ef58 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.142744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.147871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.148761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.149045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.149315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.156778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.158266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:49.173104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:49.173992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:49.174868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 311/585] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:49.175370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.196029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8079f3-1f56-4528-8ffe-e5bed686fc62 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.232534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.233275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.233561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.233848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.241491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.243218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:49.258120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:49.259270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:49.260943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:49.264497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 275/586] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:49.265358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.286592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc272-eefe-44fa-b110-a45d4fd2c47f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.325949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.326829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.327111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.327393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.338065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.339497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:49.356926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:49.358410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 312/587] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:49.359445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.370856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.379755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95405fd7-145f-40ad-9917-536bba4c295d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.395749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.420968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.421632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.421912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.422187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.422969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:49.423272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.426908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.441390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 276/588] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:49.446924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.470235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276705-ca33-4a2b-a17c-8648d0396ad9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.506579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.507153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.507456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.507706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.508516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:49.508826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.510788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.524265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 313/589] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:49.525347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.555258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b008a-901b-4210-83ad-764f0e819f99 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.589576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.590231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.590507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.590857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.591601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:49.591937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.593346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.605004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.611229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 277/590] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:49.615805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.642511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cd626-33dc-4218-b045-a46f9f1bba50 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.648792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.678453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.679129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.679411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.679688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.680491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:49.680805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.682695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.699674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 314/591] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:49.699674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.722928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd64e-8deb-4f0a-8421-fd392e3b86b0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.760381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.761109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.761385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.761660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.769085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.770519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:49.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:49.785795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:49.786732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 278/592] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:49.787846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.810074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef2b2c-bf12-459d-871c-db8d7acf3992 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.836154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.844145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.845032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.845307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.845577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.860099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=e441b994217b4b81be762246ef726545, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.861535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:49.875419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 315/593] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:49 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:49.876270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:49.906767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:49.919467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadea3-4c56-4ddf-b981-439b16cb01a3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:49.969283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:49.971254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:49.971618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:49.972059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:49.988472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:49.990157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.008326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d9a2-3e32-4560-8eef-1b8ad7c0657e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:50.017448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:50.018560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:50.019676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:50.019676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:50.019676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:50.019676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:50.021513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 279/594] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:50.022534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.051660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.052823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.053104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.053381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.061576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.063797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:50.080544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:50.081496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:50.082399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:50.083281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 316/595] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:50.083945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.134061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.160813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ef5aa-59fc-4485-ba23-e014966d01c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.198296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.199073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.199362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.199743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.200630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:50.200945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.202375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.202484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3a815-ee80-45dc-b392-9280be9528c8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.223724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 280/596] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:50.225097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.228747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.240646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.241545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.241993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.242775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.243801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:50.244151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.245551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac702b-5821-4fa2-8361-dd11d4ce4f4b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.260407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 317/597] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:50.262102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e922-68d3-42ee-bb82-be71ec9072d7 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.262703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e922-68d3-42ee-bb82-be71ec9072d7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.263912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e922-68d3-42ee-bb82-be71ec9072d7 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.265640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 318/598] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:50.266461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31e922-68d3-42ee-bb82-be71ec9072d7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.334356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f19fc3-0f9f-4710-b073-427d0d510ff9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.372785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.375658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.376489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.376875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.377200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.377639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31e922-68d3-42ee-bb82-be71ec9072d7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.378004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:50.378312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.379930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.400039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 281/599] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:50.400039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.415737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.416676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.417028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.417394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.430830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.433007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-0537922c-6e57-423d-ac74-b1ed686f7f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:50.452272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:50.453218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:50.454098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:50.454098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:50.454098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:50.454098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:50.454098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 319/600] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:50.454999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.454999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.455250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.472057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:50.473224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:50.476371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.520069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13691fc7-9e7f-412e-897f-ad859e9536dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.548581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 320/601] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:50.549188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.560762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.562774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.563224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.563647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.572762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.574216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:50.589488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:50.590400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 282/602] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:50.596858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.640068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.659222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40d57c-a4b3-4037-a1a8-37b974bb2a6b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.695013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.695934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.696218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.696495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/1a2388c351814d5193403a4b0d00b0af/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.707679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5c9b9-4181-4623-b01e-c9c50178e9d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.711607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=1a2388c351814d5193403a4b0d00b0af, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.713111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.725448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 321/603] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:50 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/1a2388c351814d5193403a4b0d00b0af/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:50.733585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.738453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.755558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.756540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.757177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.757508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.758268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:50.758578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.760747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.777889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 283/604] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:50.778784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.849597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a16a-04c3-4a13-bfab-0f54de3b3c51 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.883898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.894154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.895495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.896272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.896878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.911117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:50.916654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc4774-133f-435a-a3bc-c8637e18e07e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:50.920367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:50.921290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:50.922122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 322/605] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:50.922817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:50.956221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:50.956996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:50.957278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:50.957604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:50.958348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:50.958663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:50.960192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:50.972929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 284/606] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:50.973771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.024168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.052166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af893d3-6c38-404e-a32f-ae08c0fb5212 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.091463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.092604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.093195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.093667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.101516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.102480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da8d9f-ca72-4c46-ad1b-3ab7220f7026 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.103977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:51.120570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:51.121711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:51.122798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:51.127918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 323/607] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:51.129903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.148032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.149145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.149501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.149848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.151470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:51.151877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.154130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.165260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 285/608] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:51.166658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.193096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.250472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c122e2-b114-4ba8-ae7c-226e960a562f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.288200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.289476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.289982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.290603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/51875df6e7b1444cb31e793aa8727363/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.299731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418c79-a248-499f-8aae-a6f184096797 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.302316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.310581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=51875df6e7b1444cb31e793aa8727363, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.312419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.331538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 324/609] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:51 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/51875df6e7b1444cb31e793aa8727363/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:51.332819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.348654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.349860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.350343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.351389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.352560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:51.353182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.355544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.373353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 286/610] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:51.381302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.550427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.554567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a12d0-60eb-4ab1-8b7b-8179fe16f7d6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.583444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d08c-1010-4ca5-9f67-39ed43c34791 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.601767 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.603105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.603632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.604133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.616235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.618404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.625243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.626142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.626425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.626701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/29a153f06f9e4b459a413f2f936c1f9f/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:51.639940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:51.640838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:51.641720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:51.641720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:51.641720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:51.641720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:51.643550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=29a153f06f9e4b459a413f2f936c1f9f, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.644302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 325/611] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:51.645183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.652868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.664685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 287/612] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:51 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/29a153f06f9e4b459a413f2f936c1f9f/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:51.776556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a0dde-ec01-4bc3-9987-49c47c2628aa None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:51.834338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:51.835312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.835747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.836190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.837086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:51.837544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:51.839673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:51.857339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 326/613] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:51.863408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:51.892951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.995821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c3202-3a8e-4a36-91ce-7a0f337848c1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.036985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.037802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.038130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.038411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.046446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.047924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f9b8f048-e57e-4bda-9bbb-c42c2938e882 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:52.074253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:52.075264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:52.076238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:52.080123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 288/614] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:52.081383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.112218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.179971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d3e6-021f-4e16-8b77-8ab5d6a0f811 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.180308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d3e6-021f-4e16-8b77-8ab5d6a0f811 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.180540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d3e6-021f-4e16-8b77-8ab5d6a0f811 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.182744 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 289/615] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:23:52.190338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.190618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.190891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.219433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:52.220646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:52.236734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93654108-b868-4273-aaf6-a4dcbe62a79b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.290534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.291732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.292032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.292302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.293150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:52.293460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.294886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.313748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 327/616] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:52.319680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.399435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.409144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 290/617] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:23:52.432861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01fab5-25a4-4f00-a72b-832f40e8d8e2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.438359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.441893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.473943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.474802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.475272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.475381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.486773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.488292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:52.507855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:52.508816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:52.510267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:52.510267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:52.510267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:52.510267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:52.510267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 328/618] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:52.515805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.556062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de996ad-c2f3-4b44-9381-b21450661b60 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.608458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.610820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.611886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.612531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.636589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:52.644898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:52.645838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:52.646835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:52.661447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 291/619] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:52.666998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.699347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105cc50-9b78-4e43-8cf4-71ebfcd87b7d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.748787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.769865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.770069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.770350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.770627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.771429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:52.771815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.775185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.789882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.804402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 329/620] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:52.808368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.850867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55702-f703-46ae-8a9e-fab3de81a4ab None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.887243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.888321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.888814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.889305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.890347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:52.890876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.892559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.902881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 292/621] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:52.908663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.978019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e12261-4390-406f-8c08-0c96c266e5ce None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.046319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.047530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.048064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.048630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.064447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.066195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.069114 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4f341-0726-47a6-826c-504beedd3493 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:53.095166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:53.096184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:53.097085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:53.112052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 330/622] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:53.120138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.122593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.135120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.136193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.136715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.137216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.151168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.152703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fab70637-e5db-4219-bdf6-7e33d30ced5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:53.172416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:53.173328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:53.174217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:53.174217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:53.174217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:53.174217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:53.178340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 293/623] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:53.179883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326aeb-9e90-4d5a-97dc-116e5dc8848f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.180177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326aeb-9e90-4d5a-97dc-116e5dc8848f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.180456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326aeb-9e90-4d5a-97dc-116e5dc8848f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.186449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 294/624] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:53.186449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac326aeb-9e90-4d5a-97dc-116e5dc8848f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.194287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.261172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87868d9f-9cbb-46e1-ba38-e9cca42598f8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.298207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.299026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.299429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.299833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.300818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:53.301238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.302757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15613b2-6f13-4e6e-ae3f-de83d6ec69bd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.316091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 331/625] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:53.317552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.318156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.318718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.318855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac326aeb-9e90-4d5a-97dc-116e5dc8848f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.337827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:53.339117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:53.364875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.365736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.367102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.367478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.368728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:53.370784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.372968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.389079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 295/626] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:53.390068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.431786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 332/627] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:53.431786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.442848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.458304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.528650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5143472-3469-45b2-a9a0-15480568e5f8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.552307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5bbb3-aa7f-4726-a836-805936fbe846 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.565987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.567045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.567329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.567602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/6bd7c157dada45049506a091a97079bf/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.584351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=6bd7c157dada45049506a091a97079bf, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.586743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.595620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.596551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.596920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.597263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.602903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 296/628] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:23:53 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/6bd7c157dada45049506a091a97079bf/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:23:53.604357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.606168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.607819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:53.641813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:53.642875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:53.644005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 333/629] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:53.649490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.727866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.753498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bde98-9e03-43e2-a48b-3b5c749abb83 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.821500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:53.824532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.825353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.825635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.825915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.843718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442beaf9-b789-4ee3-834a-82356fddd051 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:53.848861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:53.856892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:53.857878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:53.858721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 297/630] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:53.865695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:53.891565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:53.892274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:53.892556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:53.894754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:53.895629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:53.896015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:53.897952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:53.918243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 334/631] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:53.924419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.028797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124a203-560d-4314-8df6-2984dd3aec35 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.065992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.066898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.067208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.067875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.068951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:54.069261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.071666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932798a2-847a-4f80-a490-6fa8db682717 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.074082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2b6e-3b80-42e7-8e4b-d98799d4fc3c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.082403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 298/632] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:54.084267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b947c-13a8-431a-a1d1-2b4dccfdf7f3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.084553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b947c-13a8-431a-a1d1-2b4dccfdf7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.084830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b947c-13a8-431a-a1d1-2b4dccfdf7f3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.088329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 299/633] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:54.089110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b947c-13a8-431a-a1d1-2b4dccfdf7f3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.108245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.126992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.128232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.128807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.129481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.151159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.203330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.204819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157bfa4-9e54-433f-b62c-b50610edcd92 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.212344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b947c-13a8-431a-a1d1-2b4dccfdf7f3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.246980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.248116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.248561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.249001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.260174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.261724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-04888df1-8c23-46b5-82e6-067e5a3364b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:54.278886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:54.280041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:54.280981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:54.280981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:54.280981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:54.280981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:54.284109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 300/634] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:54.285253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.285601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.285935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.302608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:54.303807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:54.309991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 335/635] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:54.311503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e3345-d183-47bb-b0d8-04be655b08e2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.312333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e3345-d183-47bb-b0d8-04be655b08e2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.312739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e3345-d183-47bb-b0d8-04be655b08e2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.314740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 336/636] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:54.315788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e3345-d183-47bb-b0d8-04be655b08e2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.362347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.374060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 301/637] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:54.374998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.383404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.446209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e3345-d183-47bb-b0d8-04be655b08e2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.483512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3a738-b6e0-4b90-9d91-a0b23f02a3d4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.483908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.484568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.484851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.485124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.485841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:54.486149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.487514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1559-00a3-45ce-bdbe-789d9bdc0549 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.499939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 337/638] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:54.501190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.501478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.501753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.517417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:54.518600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:54.526562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.527719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.528102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.528381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.539098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.540566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:54.560040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:54.560993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:54.561879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:54.561879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:54.561879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:54.561879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:54.561879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 302/639] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:54.562669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.592191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 338/640] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:54.593127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.619630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.641766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.679738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4c9ac-3014-4d01-bb83-c04d1915b2b0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.703804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffe24e-d82d-48e3-b48c-b4719444a435 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.715379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.716210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.716492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.716762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.727306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:54.735517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:54.742132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 303/641] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:54.743284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.744643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.744643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.744643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.744643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.751554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.753045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:54.767656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:54.768737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 339/642] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:54.769689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.858997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3901f-ca91-417a-8781-0a4fd112fc9d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.861578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.875153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.879117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855cb2-3552-4060-8ad1-dc468f0eedd9 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:54.894140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.894810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.895098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.895373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.896123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:54.896436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:54.897796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:54.910041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 304/643] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:54.910831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:54.921842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:54.922858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.923341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.923840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.934486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:54.942534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:54.943459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:54.944385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:54.951306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 340/644] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:54.952337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.017004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c87ad-b492-4690-93e6-a4658fd8fe73 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.055574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e97c6-678e-408c-9b6c-98ba6668d556 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.056110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.056777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.057216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.057497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.058972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.059283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.060669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.075309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 305/645] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.075309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.088921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.089592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.089875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.090153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.090986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.091294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.092739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.107723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.111971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 341/646] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.112787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.132492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.193014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1b53-1cae-45ae-8b71-ca5995e99fd1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.221608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f48ce-394e-4f53-b3ff-956144896d54 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.227826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.229184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.229582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.229860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.236727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.238208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:55.252970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:55.253888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 306/647] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:55.255286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.265110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.265778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.266055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.266609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.267269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.267572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.269078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.278918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 342/648] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.279739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.350705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.367268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.367438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a257-c59c-4343-ab99-8cc45dd4adf6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.388716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0b8d-8fb9-45d2-96c8-0f2ed039a6de None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.406596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.407392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.407666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.407963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.418426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.419980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.428820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.430034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.430494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.430912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:55.438940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:55.439878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:55.440807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:55.440807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:55.440807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.440807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:55.440807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 307/649] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:55.441496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.517739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.519323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.553122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d144887-a043-44f2-949f-b9c61678ebce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.591221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.591920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.592236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.592475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.593582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.593582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.594960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.602325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.604507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.608951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 308/650] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.610073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.631551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 343/651] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:55.637973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.720530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499acb3-3543-48dc-b72d-63bfa99a5522 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.738372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc1c6-3cc5-49b0-bd80-5a3ae09b1c85 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.756064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.756869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.757152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.757428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.768357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.769784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.773022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.773680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.773962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.774238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.774956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.775334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.777146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:55.786688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:55.787568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:55.788494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:55.788494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:55.788494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.788494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:55.788494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 309/652] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:55.788765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 344/653] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.788834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.789126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.833061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.848093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.891236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea285f-871d-44fd-9e43-5f2c0aaef1f0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.899229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761153c-7ee0-4449-ac9c-3ca496bf9270 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.924244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.925039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.925321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.925599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.932231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.932876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.933158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.933432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.934342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:55.934650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.936089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca3e3-3e9f-4958-8979-3dd017a8d035 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.936656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.938103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.941871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 310/654] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:55.943167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912bfc-8b7d-49ac-9ccc-b8e3166dd4be None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.943543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912bfc-8b7d-49ac-9ccc-b8e3166dd4be None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.943875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912bfc-8b7d-49ac-9ccc-b8e3166dd4be None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.945131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 311/655] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:55.945832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17912bfc-8b7d-49ac-9ccc-b8e3166dd4be None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:55.972753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:55.973704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:55.974560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:55.974560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:55.974560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:55.974560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:55.974560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 345/656] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:55.980517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.066647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17912bfc-8b7d-49ac-9ccc-b8e3166dd4be None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.083406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.093370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.102912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b2174-33de-4b3c-be21-2fc9a62c9e2c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.110116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.111245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.112250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.112250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.112540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:56.113407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.114902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156f4b-b846-4d0a-8a67-e00ac462cf50 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.128771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 312/657] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:56.130402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.130692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.130962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.143720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.144545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.144873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.145095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.149428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:56.150754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:56.153987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.155463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:56.172496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:56.173471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:56.174603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 346/658] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:56.175321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.227072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 313/659] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:56.227883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.298379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cf6f-7cea-4164-8d5a-8f00a8b349b5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.332761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001ac09-e610-4f36-b229-3e6a6c42c7a3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.336108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.339260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.340231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.340700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.341156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.342086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:56.342585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.344170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.357360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 347/660] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:56.358677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.371982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.372996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.373322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.373594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.377861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.381770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.383856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:56.400241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:56.401219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 314/661] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:56.402185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.473772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b1b53-579a-49b5-8fa2-39903a82647d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.509962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.510553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.510837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.511115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.511428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77249-c07e-4aa1-b232-582d6d0947eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.511979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:56.512288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.513716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.536483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 348/662] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:56.536483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.548758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.549890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.550431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.550614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.561339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:56.571248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:56.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:23:56.573812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:56.574990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 315/663] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:56.578306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.605672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.622105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.655661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdfc3a-a349-4784-b587-7c13fefa4f13 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.687404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df212b-0cbb-4376-adbc-d9c313268cf4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.691788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.692601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.692886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.693164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.700772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.702312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:56.716743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:56.717663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:56.718552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 349/664] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:56.719623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.733049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.733722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.734005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.734280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.735125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:56.735435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.737081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.756297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 316/665] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:56.757105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.831266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bde3-62da-4639-8d55-e11ac848fbb1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.842970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.861225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:56.863037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10f4b-a9b6-46d9-8ff2-6ca18207a516 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:56.871670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.872669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.872962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.873242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.899631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:56.900889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:56.901179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:56.901746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:56.902211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:56.902517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.904447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:56.920678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 317/666] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:56.920678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:56.953546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:56.954991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.018715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866fa2a-360b-4d1e-a42a-dbab13293492 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.051879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.052540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.053816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.054101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.054881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:57.055190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.056686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.063587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 350/667] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:57.064493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.074076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 318/668] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:57.075111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.079996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.105752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.181459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd2e1f-9650-4725-8c0d-d2b57c32788f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.191588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3db79-8fdb-4f71-9fa3-648578e411f7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.218111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.218914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.219195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.219467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.230193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.231639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.234675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.235497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.235954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.236406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:57.249335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:57.250376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:57.251306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:57.251306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:57.251306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.251306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:57.251306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 319/669] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:57.251612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.309658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.321118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.322637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.353098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.357924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b664a-3e44-49c6-a511-001bbd79871a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.395476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.396306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.396587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.396861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.407604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.409071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:57.427418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:57.428359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:57.429242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:57.429242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:57.429242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.429242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:57.433149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 320/670] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:57.433947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.439744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 351/671] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:57.440793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.541835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6ed86-caa0-48d6-895f-8a916140a698 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.547017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bf61-5bd7-4f3c-a91c-ef5bc7ad6b2b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.551898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.580671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.581341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.581621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.581890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.582732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=1a2388c351814d5193403a4b0d00b0af) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:57.583044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.584524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.587172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.591889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 321/672] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/role_assignments?user.id=1a2388c351814d5193403a4b0d00b0af&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:57.592833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.606050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.607136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.607625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.608195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.609929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:57.610482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.613016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.631127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 352/673] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:57.632567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.727474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3f4d-a2e3-406a-88ed-e15462360039 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.764639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae212d-0b48-4588-a4b2-a01b644fb2d7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.776472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.778021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.778508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.779148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.780087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:57.780394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.782336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.797281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 322/674] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:57.797965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.806632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.807437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.807721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.808041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.825752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.827222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.845444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:57.850917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:57.852059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:57.852946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:57.852946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:57.852946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:57.852946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:57.852946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 353/675] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:57.853459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:57.908833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.926192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bc98f-278f-4cee-8db8-4f13e3950580 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.964538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:57.965890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:57.966683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:57.967181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:57.977248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:57.979438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:57.983731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351598-11b7-47c3-865d-98158c0f69ab None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:23:57.999650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:58.000720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:23:58.001642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:58.007795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 323/676] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:58.008940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.022907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.024301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.024682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.025453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.034449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.036807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:58.055397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:58.056914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:58.057788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 354/677] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:58.058270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.127176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.136872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083650f-025c-4915-a9c4-70c2b57cae1d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.167621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac918d6-57fb-4f7e-b2ec-e2fef29756bd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.169131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.176146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.176941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.177224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.177499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.178271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:58.178686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.180193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.194122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 324/678] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:58.194976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.210859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.211965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.212260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.212562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.213430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:58.213749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.215194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73ee4d-5c28-43b6-b922-789b65a452dd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.232950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 355/679] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:58.234244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024fa3c-2db7-433e-b571-12c7ef77fb8b None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.234535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024fa3c-2db7-433e-b571-12c7ef77fb8b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.234811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024fa3c-2db7-433e-b571-12c7ef77fb8b None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.239812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 356/680] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:58.239812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024fa3c-2db7-433e-b571-12c7ef77fb8b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.314131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1ab02-2594-4791-9dcb-0459ac1b5383 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.344413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024fa3c-2db7-433e-b571-12c7ef77fb8b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.350549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.351494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.352043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.352502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.354134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:58.354453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.356032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.369933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 325/681] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:58.370743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.386697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.387258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.388603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.389336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.390061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.398111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.399850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-054cf5a3-81c6-4296-ab31-930ec680fed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:58.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:58.416679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:58.417714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:58.418173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 357/682] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:58.419008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.419293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.419613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.443332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:58.444553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:58.459851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.483181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d337ba-5a3b-49e2-a00c-89240558a28b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.516852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 358/683] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:23:58.519601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.520371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.520643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.520899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.528740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.593566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.595009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.610308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.639212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b881c4-4362-449e-bb2b-ce91597c7f35 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.675450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.676072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.676362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.676630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.677465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=51875df6e7b1444cb31e793aa8727363) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:58.677780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.679201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.695799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 359/684] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/role_assignments?user.id=51875df6e7b1444cb31e793aa8727363&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:58.695799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.714278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.725351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 326/685] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:58.726358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.820810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec41f6-d50a-4933-b594-dbaf32656606 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.871073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.871931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.872219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.872659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.874532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:58.875100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:58.876886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:58.879736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db47015-5b42-486d-be8d-c48ada512e2b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.892629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 360/686] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:58.893477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.895180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.921619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:58.922778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.923257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.923490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.935602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:23:58.944717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:58.945677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:23:58.946515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:58.950873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 327/687] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:58.951790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.987640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.025164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdad30e-42b7-4b16-bf42-8289c2fcb06f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.058664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2040f-577b-4bb9-829e-c8756f7eb914 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.075301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.076596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.077058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.077785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.092057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.094503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.097464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.098509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.098862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.099228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.100173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=29a153f06f9e4b459a413f2f936c1f9f) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.100552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.102037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.109970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 328/688] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/role_assignments?user.id=29a153f06f9e4b459a413f2f936c1f9f&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:59.110944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:59.117175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:59.118423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:59.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:59.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:59.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:59.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:59.121466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 361/689] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:23:59.130218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.175947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.296881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14de05-c34a-4255-8add-ceb750fbac94 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.322535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32df52-245a-4133-99b8-6c3fbb85584f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.352899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.353612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.353791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.353889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.354158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.355161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.355939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.357657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.366048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.366724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.367097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.367416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.369681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.369998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.371385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.371899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 329/690] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:59.374005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.385783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 362/691] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:59.391234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.480422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.486174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429631c1-0b54-4328-8ea0-770ba8bdf1dc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.516917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88634-28d9-44f5-a1dc-923b6d6f0d68 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.525116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.526284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.526772 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.527282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.538974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.540820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.554332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.555061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.555347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.555627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:23:59.561937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:59.562875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:59.563811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:23:59.563811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:23:59.563811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:23:59.563811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:23:59.564050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.564944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.569999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 330/692] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:59.572696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:23:59.580391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:23:59.581305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:23:59.582604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 363/693] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:23:59.587633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.640473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.694137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5797-2f4e-4384-968a-d2379f8a0840 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.699041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac3c3a-2ad1-4af1-8755-e55a77e509b7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.729933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.731043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.731456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.731735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.732501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.732809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.734183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.735283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.736166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.736601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.736948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.737814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.738193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.739705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.748572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 331/694] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:59.757199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.760743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 364/695] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:23:59.772543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:59.785033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.940263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504ba2-b3e1-46ab-8db0-d9d1ee18a112 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.943774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f2986-d5d6-4ce2-808f-05bee343e0d5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:59.948607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.977779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.979091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.979368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.979639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.980599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=6bd7c157dada45049506a091a97079bf) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:23:59.980911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.982699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c4933-a6ac-4a08-ac27-be63f626615c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.984815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.985620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.985946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.986229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.990141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 365/696] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/role_assignments?user.id=6bd7c157dada45049506a091a97079bf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:23:59.991500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e473c-0cb0-463a-89b2-113f20712c6b None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.991811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e473c-0cb0-463a-89b2-113f20712c6b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.992096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e473c-0cb0-463a-89b2-113f20712c6b None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.994891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.996418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.998540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 366/697] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:00.000319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.000612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.000887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:00.015278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:00.016323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:00.018343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 332/698] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:23:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:00.019815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:00.021248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:00.023785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.101021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 367/699] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:00.108182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.128931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.142244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495a5e-7df6-4517-b668-af71da796ff9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.179875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.180590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.180875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.181154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.182548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:00.182860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.184433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.204688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 333/700] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:00.210095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.238622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.242513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893a997-9f70-4cba-87a7-7f28066d30bb None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.307611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.309286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.309575 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.309853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.323808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.325543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.330276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667bc51-2f83-47b4-bb64-a778cd3e1604 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:00.349125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:00.350063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:00.351103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:00.351103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:00.351103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:00.351103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:00.351103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 368/701] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:00.363793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.365190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.377271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.377932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.378216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.378489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.468639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.470463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.482315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6718-7b83-4fb1-a9bc-3e77473fde0c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.529554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.530230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.530667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.530963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.532116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:00.532563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.534229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.555253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 369/702] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:00.561016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.577327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.606522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 334/703] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:24:00.611937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.645555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.668354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41be9-ef08-478a-913b-aaac7fb93d16 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.703547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.704269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.704553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.704828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.705549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.706977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.720979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a000b56-8654-4241-8a2e-d0327740769d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.751403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 370/704] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:24:00 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:00.764213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.765074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.765454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.765734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.778676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:00.780509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:00.801961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:00.804155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:00.806375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:00.806375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:00.806375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:00.806375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:00.806375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 335/705] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:00.816945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.933279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.006319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af2554-98c0-4eb4-b135-715296cf4163 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.053281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.053955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.054261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.054543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.055286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:01.055594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.057632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a4faa-f8df-4390-9c62-4d13fd537baa None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.078010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 371/706] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:01.083485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.131527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7784f-9e54-43bb-9370-a1b77f3512e6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.131893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7784f-9e54-43bb-9370-a1b77f3512e6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.132084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7784f-9e54-43bb-9370-a1b77f3512e6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.134405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 372/707] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:01.141128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.141592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.142302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.161365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:01.162832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:01.184109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.205104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850111d3-b2a3-47e8-9e1e-f577220fc74b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.239868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 373/708] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:24:01.245000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.245832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.246119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.246395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.253514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.254969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.262496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:01.269660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:01.270594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:01.272254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 336/709] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:01.276669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.355254 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.402485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02947a53-6155-4eec-b811-a1b6e6a50fdb None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.405067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4facc-9321-46fa-98aa-94c7c5b6c9d7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.441146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.442339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.442770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.443140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.443974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.444132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.444948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.445427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.445759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.445999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.447005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:01.447476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.449074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.460108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:01.460839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:01.462011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:01.463099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:01.465767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 337/711] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:01.471804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 374/711] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:01.473014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.476848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.580027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc618683-1252-4653-b16b-aee24502e81c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.616284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.617265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.618018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.618557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.620055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:01.620555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.622105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.627364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 338/712] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:01.632660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.708165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.719325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18f7ee-10a7-4110-bbe1-8f72550710de None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.741235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec376ac-21b9-4c02-8fa8-cfcda1d7a8a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.766799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.777709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.778632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.779101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.779578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.780489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:01.780997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.782559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58739e6-32f4-4015-8b4f-9c1f6b2411f5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.789870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 339/713] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:01.793856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.794796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.795263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.795714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.796797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:01.797302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.798999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595112a-4ce5-430e-844c-7c1adcf40fa4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.814539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 375/714] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:02.049983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b0c33-2a83-439c-989f-5f1effe04444 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.050542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b0c33-2a83-439c-989f-5f1effe04444 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.051000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b0c33-2a83-439c-989f-5f1effe04444 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.052806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 340/715] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:02.059149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.060062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.060550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.080077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:02.082357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:02.177989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 376/716] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:02.209924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b0c33-2a83-439c-989f-5f1effe04444 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:02.358538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b0c33-2a83-439c-989f-5f1effe04444 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:02.390395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:02.398851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:02.399662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.400003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.400500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.401164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:02.402592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:02.416539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:02.417527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:02.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 341/717] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:02 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:02.426062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:02.553144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6c268-e831-4d31-8bbd-396cccaf47b0 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:02.571455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:02.599270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:02.600762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.601043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.601318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.602029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] RBAC: Adding query filter params (name=network) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:02.602335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:02.604597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c4e56-ca1d-4d68-8af5-37425001927a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:02.609888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 377/718] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:02.615220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:02.748259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27ceef-ff64-4eff-9327-09f27c2c84bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:02.796439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:02.797502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.797982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.798453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.799352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:02.799886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:02.801474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2518a-cdb5-4bfc-bc5c-66b1bb48f02e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:02.807369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 342/719] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:02.921458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92027-8c87-4abf-be63-2b0f54b6bb16 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.921754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92027-8c87-4abf-be63-2b0f54b6bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.921942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92027-8c87-4abf-be63-2b0f54b6bb16 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.923543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 378/720] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:02.929123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018823-e87e-459d-9374-7c6562474165 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:02.929738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018823-e87e-459d-9374-7c6562474165 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:02.930240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018823-e87e-459d-9374-7c6562474165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:02.961048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-35018823-e87e-459d-9374-7c6562474165 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:02.962679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-35018823-e87e-459d-9374-7c6562474165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:03.028764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:03.046372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 343/721] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:03.070399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd92027-8c87-4abf-be63-2b0f54b6bb16 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:03.186358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd92027-8c87-4abf-be63-2b0f54b6bb16 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:03.239995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:03.240807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:03.241094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:03.241370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:03.254806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:03.263347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:03.264291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:03.265121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 379/722] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:03.270448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35018823-e87e-459d-9374-7c6562474165 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:03.431795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35018823-e87e-459d-9374-7c6562474165 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:03.480517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:03.481116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:03.481401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:03.481676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:03.482423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:03.482729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:03.488175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d405a-0c50-4521-b6fa-578829d3547e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:03.497423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 344/723] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:03.502877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:03.578057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:03.652295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727e5a6-45b0-4dfe-8913-6fc4fd7af560 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:03.658715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601196-50cd-420e-9f32-30a3b6621e83 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:03.659007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601196-50cd-420e-9f32-30a3b6621e83 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:03.659286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601196-50cd-420e-9f32-30a3b6621e83 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:03.660678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 345/724] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:03.665828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:03.666116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:03.666395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:03.693425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:03.694607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:03.716958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:03.717766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:03.718044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:03.718320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:03.738132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:03.739513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:03.768681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:03.769686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:03.770574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:03.770574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:03.770574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:03.770574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:03.770574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 380/725] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:03.780868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:03.781752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:03.809212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 346/726] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:24:03.855684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:03.930464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:03.937107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986d7e-1c56-4d59-b2d9-98a55afe8b8e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.001221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.001810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.002088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.002363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.003094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:04.003404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.009936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.034950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 381/727] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:04.038031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e5648-dcc0-4840-b3eb-0186d806e765 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.039262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.106456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.107321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.107605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.107901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.131003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:04.141867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:04.143492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:04.144369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 347/728] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:24:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:04.155088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.213073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acac897-05c7-422b-bf5b-bbf7d1dd263c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.236272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.281092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.282518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.283041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.284154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.297078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.299807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-c865b2dd-b8d2-46ef-af40-e60d779deabe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:04.320654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:04.321658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:04.323205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:04.323729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b575c-e9cc-4020-a924-b468eaf38aeb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 382/729] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51942-056e-4a0f-bd85-727dee2e3ef6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51942-056e-4a0f-bd85-727dee2e3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51942-056e-4a0f-bd85-727dee2e3ef6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 383/730] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:04.332698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51942-056e-4a0f-bd85-727dee2e3ef6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.353517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.366250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.367222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.367543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.368160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.369111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:04.369484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.371036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54751894-cff7-4be9-a2d4-e092d6c81bf7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.380819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 348/731] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:04.382477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.382827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.383176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.401189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:04.402509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:04.468675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51942-056e-4a0f-bd85-727dee2e3ef6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.481191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 349/732] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:04.488032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.520780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.521814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.522311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.522810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.524564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:04.525203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.527407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.533397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.557418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 384/733] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:04.557418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.626222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.629564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a10ca-7b87-4057-83cb-8aa0bb285ac7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.663149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686e79c-f9bf-4606-930c-a9843cf44d9a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.667207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.668028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.668381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.668652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.679268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.680739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:04.698189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:04.699128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:04.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:04.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:04.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:04.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:04.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 350/734] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:04.700333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.707683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.708516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.708798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.709072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.721896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d8adf8b4-b82e-4b8a-ba9a-a02b271b1999 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:04.730122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:04.731134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:04.732032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 385/735] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:04.732718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d025d0-7009-47c7-a472-21a5ca5a0654 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.733572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d025d0-7009-47c7-a472-21a5ca5a0654 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.733572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d025d0-7009-47c7-a472-21a5ca5a0654 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.738353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 386/736] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:04.738353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d025d0-7009-47c7-a472-21a5ca5a0654 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.801414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.822743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f48d-c982-4e95-a168-ae89778807d3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.841599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d025d0-7009-47c7-a472-21a5ca5a0654 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.858381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.859029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.859309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.859590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.871664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.879239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.879927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.880208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.880483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.881206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:04.881510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.882870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:04.900153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 387/737] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:04.900153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.935264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:04.937502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ff5b0-0327-4ae7-8d7d-5565c9abfb61 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.004167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c59bc-bf1b-4130-91dd-c2d3e0397833 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.028527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.039485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.040270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.041071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.041349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.042099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:05.042410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.043786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.054217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 351/738] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:05.054217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.054217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.054217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.057703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 388/739] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:05.058422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.074596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:05.076333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:05.116323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.148957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 352/740] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:05.150550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.170536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fb0ca-2f10-4849-a604-9b33ec5c8496 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.206291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.207106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.207390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.207651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.216111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.217559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:05.233062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:05.234034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:05.234969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 389/741] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:05.235549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.261993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ab9d-23f7-40b0-8f2f-7a6dd75d1f7f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.274779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.296030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.296817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.297095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.297367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.307691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.309088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:05.326214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:05.327544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:05.328663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:05.328663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:05.328663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:05.328663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:05.332076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 353/742] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:05.332743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.333453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1b0b-cd5f-4c40-a8cb-1c63a16a3409 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.346071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.368774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.370048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.370328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.370597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.380942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.382370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-63736b86-9d5e-4b2b-ae41-feaec9c863b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:05.399135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:05.400618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:05.401502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:05.401502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:05.401502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:05.401502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:05.401502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 390/743] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:05.401995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3cfda-196c-49e1-a023-866a090b2ea6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.402355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3cfda-196c-49e1-a023-866a090b2ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.402543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3cfda-196c-49e1-a023-866a090b2ea6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.404007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 391/744] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:05.404485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3cfda-196c-49e1-a023-866a090b2ea6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.437658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7bbf-122b-4233-8b0a-c6a0eabf8833 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.471186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.471877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.472164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.472442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.473168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.474533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.489840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 354/745] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:05 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:05.490828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.520528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.524393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3cfda-196c-49e1-a023-866a090b2ea6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.563428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.564062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.564349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.564625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.565605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:05.565926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.567384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.587334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 392/746] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:05.588391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.608976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.616221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e7429-587a-4391-a8a1-9638eefe20b9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.659672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.660676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.661314 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.661609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.662623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:05.662942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.664369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95ce7-4b59-4d5c-b429-71d479ceaa12 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.679058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 355/747] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:05.680385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.680678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.681057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.698721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:05.699956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:05.709639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52f625-14af-4f70-8e43-ac483311fcca None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.748738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.749451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.749730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.750010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.750749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:05.751077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.752464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.765093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 393/748] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:05.765909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.787807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 356/749] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:24:05.787807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.799076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.852765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.876285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23fa47-567e-43b0-8368-50ee040fc541 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.896258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595e2eb-293c-4044-aefb-7d37c53abebb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:05.912854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.913511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.913787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.914057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.933075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:05.933877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:05.934157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:05.934431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:05.941279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.942725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:05.957601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:05.958543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:05.959483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 357/750] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:05.960029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:05.991731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.993630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.042069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.069859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c848f4-7177-4fb3-bcc7-f0be843fcab6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.080113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.105637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 394/751] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:06.106317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.113068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.113871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.114153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.114427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.123076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.124541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:06.140355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:06.141424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 358/752] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:24:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:06.142384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.227240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0206261-50f1-4e14-ad8f-126328c65138 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.252440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d19b-9595-41e1-816e-de01319a3f5f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.265380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.266211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.266504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.267142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.268184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.269726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.286523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 395/753] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:06 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:06.287230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.299098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.313926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.315204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.315701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.316747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.318108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:06.318637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.320917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.338509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 359/754] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:06.340274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.365937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.428799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedccbc5-bd80-47fc-b55b-ec28f106c37f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.473769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.474708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.475086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.475501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.489986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653ee-fed3-4fb3-b516-64e20c7b8a97 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.491521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.494906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:06.517307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:06.518436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 396/755] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:06.519437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.534001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.534681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.534971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.535247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.536139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:06.536448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.537886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.558239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 360/756] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:06.559095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.627280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.633970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.646730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d8e8-b789-4e76-a8ab-566b35247614 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.684252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb083-7197-4456-82b2-f09a81993dac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.687698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90855736-e179-441f-8858-cce537864ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.688394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855736-e179-441f-8858-cce537864ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.688677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855736-e179-441f-8858-cce537864ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.688952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855736-e179-441f-8858-cce537864ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.689799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90855736-e179-441f-8858-cce537864ae3 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=6bd7c157dada45049506a091a97079bf) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:06.690113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90855736-e179-441f-8858-cce537864ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.691557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90855736-e179-441f-8858-cce537864ae3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.698449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 397/757] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/role_assignments?user.id=6bd7c157dada45049506a091a97079bf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:06.699361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90855736-e179-441f-8858-cce537864ae3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.721534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.722280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.722573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.722847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.723599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:06.724065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.725503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.739604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 361/758] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:06.750794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.817990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90855736-e179-441f-8858-cce537864ae3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.873207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.874027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.874313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.874588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:06.888974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:06.890383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:06.911318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:06.913387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:06.915431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:06.915431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:06.915431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:06.915431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:06.915431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 398/759] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:06.921516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:06.932761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.935442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faebf-57dd-4d91-99f3-77a3927dbf36 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:06.954258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:06.992181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:06.993628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:06.994109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:06.994603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.001759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.003513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:07.018405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:07.019341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:07.020256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:07.022743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 362/760] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:07.028871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.098332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2407583-c24b-4d47-9c81-2964cf89a6d8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.136676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4ed3-dd1e-4933-9c43-4fc2c7bd59b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.167912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.168512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.168800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.169080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.169817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:07.170128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.171623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.174827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.175721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.176231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.176695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.177725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:07.178237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.179928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.193151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 363/761] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:07.198497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 399/762] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:07.199798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.203832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.233160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.311691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddd15-6331-4327-9315-ed9879e5401d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.347223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.348714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.349258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.349752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.350780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=bd93f75ca8ee4da0b1f1d89ab59b1733) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:07.351287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.352924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.358516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.361915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 364/763] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:07 2022] GET /identity/v3/role_assignments?user.id=bd93f75ca8ee4da0b1f1d89ab59b1733&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:07.441838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39199f9-a6bd-481b-a910-9a0c98b86a10 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.493960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.495203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.495681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.496198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.503589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.505349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:07.520724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:07.521685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:07.522567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:07.525341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 400/764] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:07.532962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.634309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.644892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfacf3-2170-4983-90ce-75d21694731c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.687252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.688295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.688775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.689261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.690240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:07.690751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.692392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c496-b0a0-4077-b3b8-888219fb8a0d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.711815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 365/765] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:07.718203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.842853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c3ec3-c51e-4e8d-9c7b-a9d4a881e980 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.873133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.888803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.889705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.889990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.890265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.892290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=e441b994217b4b81be762246ef726545, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:07.893727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6241994-ba27-4624-8460-78ffa1c7a2cf None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:07.912852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 401/766] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:07 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:24:07.937463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c8287-c90c-4357-9ca0-d3a25183369b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.938105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c8287-c90c-4357-9ca0-d3a25183369b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.938579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c8287-c90c-4357-9ca0-d3a25183369b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.940605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 366/767] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:07.946073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.946356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.946643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.968320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:07.969627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:08.058581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 402/768] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:08.093333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c8287-c90c-4357-9ca0-d3a25183369b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:08.228389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c8287-c90c-4357-9ca0-d3a25183369b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:08.265069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:08.265883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.266161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.266433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.267580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:08.269759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:08.283870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:08.284851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:08.285666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 367/769] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jul 20 15:24:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:08.292927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:08.433750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed863f-518a-4f32-a73f-3df5e69e7912 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:08.461081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:08.485464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:08.486066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.486348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.486618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.487319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:08.487622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:08.489063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f77af-ce25-42c8-b36f-6c59d858ffb7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:08.499776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 403/770] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:08.505440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:08.530425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:08.636043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82ebf1-76e5-4b31-b6b7-f39567041561 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:08.646959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13203ef-7e50-407f-923d-a046812127e5 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.647234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13203ef-7e50-407f-923d-a046812127e5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.647515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13203ef-7e50-407f-923d-a046812127e5 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.649045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 404/771] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:08.657304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.657589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.657870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.680882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:08.681590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.681875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.682154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.682862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:08.683167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:08.684704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:08.690061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 368/772] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:08.692816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:08.693998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:08.696554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:08.827513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 405/773] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:08.850182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:08.861936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:08.904590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b89ea-a6b3-4c9b-8716-852e019ca17b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:08.978272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:08.978941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:08.979227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:08.979499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:08.982613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] RBAC: Adding query filter params (interface=public, service_id=819341ab9f1847dea8750d734a419c27, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:08.982931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:08.984348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.002072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 369/774] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:08 2022] GET /identity/v3/endpoints?service_id=819341ab9f1847dea8750d734a419c27&interface=public®ion_id=RegionOne => generated 194 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:09.008031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.046965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb023847-b0c9-462a-89be-a2edb8671faa None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.073110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.131430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.132584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.132917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.133199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.133870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.135309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.162287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943735dc-9154-4741-acce-bd316f4b23ae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:09.164371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:09.165317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:09.166214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:09.172257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 406/775] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:08 2022] GET /identity/v3/services/network => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:09.177908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.206807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.207481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.207774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.208059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.208773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.210166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4e266-3355-43e0-81e6-d078990e63b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.216895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 370/776] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:09.240340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9282-902c-46e0-ae72-1a61d2f54282 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.240648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9282-902c-46e0-ae72-1a61d2f54282 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.240939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a9282-902c-46e0-ae72-1a61d2f54282 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.242616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 371/777] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:09.249097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.249389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.249640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.281067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:09.282311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:09.284915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.394224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c14ca9-45d1-4c28-b844-39d25c139fd1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.425627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 372/778] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:09.435285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.439944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.440625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.440905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.441176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.441891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] RBAC: Adding query filter params (name=network) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:09.442198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.444226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.451266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 407/779] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:09.457087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.511588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.556748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942a09-81d3-49d4-85c2-fa344a665948 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.577915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c53a0e-bac7-4be0-973a-e00973bc576e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.594622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.595642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.596091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.596616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.604427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.611499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:09.614664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.615565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.616100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.616602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.617500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] RBAC: Adding query filter params (type=network) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:09.618349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:09.619694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:09.620637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:09.621565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 373/780] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:09.623137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.626830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.632080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 408/781] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:09.637736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.795678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d829e15-5895-45e6-93fb-fc1f5dfa0e34 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.820097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e9ee-3d28-4f0e-a28a-61dc71911819 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.821616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.840532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.841573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.841930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.842275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.843095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:09.843473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.845344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971462aa-d8d0-4015-8860-13d6326fc9a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.859707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 374/782] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:09.859707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62203-60c2-4b50-81e0-e57883f547dd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.859707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62203-60c2-4b50-81e0-e57883f547dd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.859707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62203-60c2-4b50-81e0-e57883f547dd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.859707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 375/783] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:09.862341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e62203-60c2-4b50-81e0-e57883f547dd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:09.866009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:09.866679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.866957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.867231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.868331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] RBAC: Adding query filter params (interface=public, service_id=e912bf7ace2844179a8b0eae26f9fe9c, region_id=RegionOne) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:09.869176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:09.871429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5639f5-3b20-4ee2-ae4c-b27e1689768c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:09.881135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 409/784] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/endpoints?service_id=e912bf7ace2844179a8b0eae26f9fe9c&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:09.882582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:09.882922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:09.883193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:09.904416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:09.905753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:09.932782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:09.979941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e62203-60c2-4b50-81e0-e57883f547dd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:09.999179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 410/785] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:09.999179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.032292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.033444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.034064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.034534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.047036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.048789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:10.067570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:10.068539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:10.069439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:10.069439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:10.069439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:10.069439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:10.076123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 376/786] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:10.077195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.106494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.114162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec1605-6374-4e5c-b11f-1e0e1b17fc60 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.161646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.162674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.163175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.163648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.165089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.167731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.175825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 411/787] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:09 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:10.176946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.193863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da19d-2d83-4894-a9c5-867b96808485 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.209404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.239797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.239797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.239797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.239797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.256803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:10.265297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:10.266283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:10.267551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 377/788] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:10.279942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.369310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0644f-4407-4f95-94f0-c9c95fcf4c73 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.405641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.406303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.407463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.407463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.407655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:10.407879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.409349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.431938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 412/789] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:10.439005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.441404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bf79c-62c4-4309-b01c-9aeb9c66c8c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.468440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.489221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.489910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.490193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.490472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.491227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:10.491540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.493256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.507458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 378/790] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:10.513897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.590327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.597792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b44c1-c4a7-422a-a180-b16847e642da None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.650444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.651316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.651698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.652114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.661262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.662872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-60460e1f-95da-452e-8a0b-402a6acb9a05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:10.680293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:10.681361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:10.682284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:10.688601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 413/791] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:10.690081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2306-1b6d-44b8-bc16-220900e47e72 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.690369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2306-1b6d-44b8-bc16-220900e47e72 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.690647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2306-1b6d-44b8-bc16-220900e47e72 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.692289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 414/792] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:10.693378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2306-1b6d-44b8-bc16-220900e47e72 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.723915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aaaf9-dd31-469f-8acb-e71736ae9c26 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.761764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.762561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.762843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.763118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.774554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.776592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e82c6250-67c0-400f-a416-85add361d77e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:10.794598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:10.795523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:10.796421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:10.796421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:10.796421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:10.796421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:10.800579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 379/793] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:10.801882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.802172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.802453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.802932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.822986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:10.824192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:10.829203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2306-1b6d-44b8-bc16-220900e47e72 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.875066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.875745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.876369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.876649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.877879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:10.878192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.879647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.900104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 380/794] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:10.901217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.906369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 415/795] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:10.908845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.935168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.036071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30bc9-08c4-4ef3-8291-0589a892d254 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.054877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daaf6c-f73e-4bc1-9fdb-80b1630db529 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.077040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.077845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.078129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.078406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.088923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.094519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.095190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.095364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.095537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.095734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.096516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:11.096701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.096846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.098371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.115293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 381/796] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:11.116237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:11.119342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.120851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.122196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:11.122196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:11.122196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.122196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:11.126125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 416/797] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:11.128043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.203102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.231219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eec368-c6ee-4b32-b8e8-7dd63da3ff4e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.266435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87293bf-25bb-4416-ba0f-c3b6ab5ce66f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.267416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.268358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.268778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.269119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.281116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:11.289089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.291272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:11.293302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:11.303451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 382/798] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:11.303451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.308223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.309453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.310597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.311107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.324796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.326256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:11.342849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.344079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:11.345396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:11.349409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 417/799] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:11.350261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.412101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.440392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33c743-8052-4c5c-95c5-0afe50b778ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.486715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36c47-00a3-4b49-a0fa-1a0f96f2988c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.491482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.492458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.493219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.494242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.495697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:11.496238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.498283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.502123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.522599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 383/800] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:11.524406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.531509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.532232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.532515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.532790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.533554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:11.533865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.535235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.549628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 418/801] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:11.550517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.640020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39202a7-b37c-44f2-8389-1800ce7f5afe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.668379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb92e45-389e-4e29-8509-737e819bc4c5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.676067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.676740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.677022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.677298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.678136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:11.678444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.679884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.685586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.701382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 384/802] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:11.702717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.720121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.721759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.722223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.722685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.734362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.736952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-00a8440f-08c7-4746-9ca3-ef9b7af2c9ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:11.763692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.764795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:11.765712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:11.771569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 419/803] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:11.771569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1e5-18cf-4349-a0b8-f69164d83d54 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.771569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1e5-18cf-4349-a0b8-f69164d83d54 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.771569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1e5-18cf-4349-a0b8-f69164d83d54 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.775849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 420/804] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:11.776954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46b1e5-18cf-4349-a0b8-f69164d83d54 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.796406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:11.855384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11bcc5-2c79-4e69-bf47-4cd368777002 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.891085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.892003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.892290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.892565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.903457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.904903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.905723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46b1e5-18cf-4349-a0b8-f69164d83d54 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:11.923297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.924265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.925162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:11.925162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:11.925162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.925162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:11.925162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 385/805] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:11.925442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:11.947950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:11.948782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.949066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.949357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:11.960866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:11.962621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-c40be571-a3f7-498a-9e71-669579fdab78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:11.982085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:11.983010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:11.983922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:11.983922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:11.983922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:11.983922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:11.985413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 421/806] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:11.986737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:11.987201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:11.987617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.006070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:12.007246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:12.025147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.047713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65565ac5-9118-43a9-bf31-05b0bdb1aaa7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.055304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.101849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 422/807] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:12.101849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.102215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.102215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.102215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.102215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.102739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:12.102739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.106955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.132993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 386/808] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:12.134303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.205121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead7240-56e5-4425-afe3-2afe166699ff None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.244873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.245587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.245936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.246228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.247011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:12.247475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.248892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.254916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a76c6-20b7-4b61-9487-786bf9f1b901 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.265667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 423/809] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:12.265667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.280113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.297399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.298079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.298359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.298641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.299369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:12.299681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.301083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.316596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 387/810] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:12.317436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.346401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.377089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14543e-dde6-4473-b61f-6b974f0d16f6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.413274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.414239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.414507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.414781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.415997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.417902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.428678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1263a-1601-405f-9ed5-de8d82f6e903 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:12.439958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:12.440919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:12.441741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 424/811] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:12.442592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.468176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.469056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.469358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.469610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/6bd7c157dada45049506a091a97079bf/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.471673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=6bd7c157dada45049506a091a97079bf, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.473209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.486082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 388/812] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:12 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/6bd7c157dada45049506a091a97079bf/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:24:12.487212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.536758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.561190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f98fdb-146c-417c-90b8-bf2430a2cdd0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.586949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.604915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b349de-7f8c-4536-9bf7-18e1a815880f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.605537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.606496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.606977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.607262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.618639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.620341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-52d66625-805c-4da5-a8fa-78661dd4990c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:12.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:12.641012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:12.641944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 425/813] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:12.642520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4cbb-6584-4511-bdba-4626224cb911 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.642764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4cbb-6584-4511-bdba-4626224cb911 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.643043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a4cbb-6584-4511-bdba-4626224cb911 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.645200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 426/814] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:12.645882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a4cbb-6584-4511-bdba-4626224cb911 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.651187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.652052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.652331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.652608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.660272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.661705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:12.675504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:12.676753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:12.677648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 389/815] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:12.678123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.762580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a4cbb-6584-4511-bdba-4626224cb911 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.782979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51d3f-18be-42e7-9795-480009033650 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:12.800908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.801510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.801797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.802066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.802776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:12.803081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.804472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8382b09-6d7d-4639-b0a0-12ab8fb5da76 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.813981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 427/816] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:12.815156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.815455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.815735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.820164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:12.820488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.820843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:12.821125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:12.821407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:12.823002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:12.823315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:12.824769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:12.834282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:12.835469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:12.843543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 390/817] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:12.844443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.849402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:12.911457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 428/818] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:12.912581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:12.970099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098c50-d8a8-4563-ab21-2b9294a812c6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.008573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.010125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.010562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.010988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.011980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:13.012445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.014091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.035949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 391/819] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:13.037056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.108197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.142629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b6e6a-18ee-4804-bd6c-da6744a55f5e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.151308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfe07a-f2db-46d3-a7b9-88451f29608a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.174004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.182215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.183190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.183632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.183956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.184676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:13.185039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.186375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.189096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.189749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.190031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.190302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.196226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 429/820] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jul 20 15:24:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:13.196226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.269696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.271146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.297406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ed42-50ab-4a58-bda0-420bc96a1614 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.332478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.333264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.333544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.333818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] PATH_INFO: `/v3/services/network` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.334461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.335192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.335826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:13.348005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:13.348995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:13.355851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 430/821] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:13.356637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.387115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 392/822] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:13.387115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.427454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.471406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c16e6-fce3-47fc-a10d-6fa4fbd6c52e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.490447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399438d-7fd4-4b7b-9a37-e4d4c7068a93 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.508236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.509154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.509443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.509724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/bd93f75ca8ee4da0b1f1d89ab59b1733/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.524299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=bd93f75ca8ee4da0b1f1d89ab59b1733, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.525812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.528173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.529025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.529494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.530052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.530857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.532495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.540235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 431/823] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:13 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/bd93f75ca8ee4da0b1f1d89ab59b1733/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:24:13.541514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.574571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 393/824] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:13.576098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.607893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.727289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e826-9bb1-463e-92ce-f7e9cf7b84ba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.750738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272359a-96ff-4c32-8635-729f54528725 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.763106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.776261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.778141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.778758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.779274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.792663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.793654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.794299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.794299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.794906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:13.795210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.796622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.797277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.799465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.802421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 432/825] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:13.807228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:13.820608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:13.821590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:13.822587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:13.822587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:13.822587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:13.822587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:13.824852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 394/826] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:13.832421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.922533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.931275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be31f22-a469-4c92-a69a-ecf026d63ca4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.964222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94505ceb-2f19-4870-b87c-a6cdcdfccbeb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.977262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.979045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.979533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.980175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.981249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] RBAC: Adding query filter params (type=network) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:13.981384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.982742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.988363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 433/827] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:13.998054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:14.007378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:14.008790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.009771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.010213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.011567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:14.012072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:14.014248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:14.030783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 395/828] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:14.036783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:14.052672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:14.123117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b159a5-e6ba-4e61-99ca-829bf0e6cc46 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:14.166817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00b24-9f3b-45eb-a144-4c3a67ad15db None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:14.189836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da968953-0870-42b1-963a-766917252e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:14.190506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da968953-0870-42b1-963a-766917252e57 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.190786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da968953-0870-42b1-963a-766917252e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.191061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da968953-0870-42b1-963a-766917252e57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.191831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da968953-0870-42b1-963a-766917252e57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:14.193362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da968953-0870-42b1-963a-766917252e57 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:14.200201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:14.215231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:14.216603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.218342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.218769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.235416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:14.237324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 434/829] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:14.239099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:14.271865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:14.273044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:14.274205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:14.280352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 396/830] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:14.288946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da968953-0870-42b1-963a-766917252e57 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:14.412218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:14.435525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da968953-0870-42b1-963a-766917252e57 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:14.481914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:14.482588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.482871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.483145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.484041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:14.484351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:14.485838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c94220-176f-430c-8c95-96efd17d5923 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:14.499258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 435/831] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:14.505647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:14.629758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd180b36-ba60-4fd5-9300-9f352450f757 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:14.668334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:14.669153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.669528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.669916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.670846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:14.671265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:14.672855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd6c50-6540-4eeb-9db6-a29b106fa449 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:14.680858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 397/832] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:14 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:14.745485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1137-4f53-47f6-81d8-7c17f187bdd5 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.745989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1137-4f53-47f6-81d8-7c17f187bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.745989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1137-4f53-47f6-81d8-7c17f187bdd5 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.747701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 436/833] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:14.753142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:14.753679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:14.753861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:14.773089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:14.774439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:14.829635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:14.851561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 398/834] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:14.878437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1137-4f53-47f6-81d8-7c17f187bdd5 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:14.987394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1137-4f53-47f6-81d8-7c17f187bdd5 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.022013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.023007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.023440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.023898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.034903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:15.042749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:15.043879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:15.044879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:15.047176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 437/835] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:15.053153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.161245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99351351-b163-48e6-9fc1-39dd7b91eb5d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.199108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.199803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.200095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.200380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.201143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:15.201451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:15.202828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:15.211816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 399/836] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:15.217270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.230952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:15.324548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f97af3-d9d3-41b4-8e7b-af504ffe0ccc None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.360363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.361154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.361433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.361702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.373539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:15.374975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:15.392662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:15.393543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:15.394423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:15.394423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:15.394423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:15.394423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:15.394423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 438/837] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:15.400742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.456852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:15.521242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed923-2aaa-419b-b2ab-5de9ab3d7998 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.555959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.557248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.557530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.557805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.558532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:15.558838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:15.560239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:15.572839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 400/838] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:15.578300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.642773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:15.688320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e383-271b-477c-8f2d-4d290f49f6b3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.722522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.723310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.723591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.723892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.732992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:15.734427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-be9a1ed4-4bd0-4b4f-aa7b-0468baca3b30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:15.749674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:15.751018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:15.751968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 439/839] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:15.758704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.848547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727404f-e111-460f-84ad-d3ea419f9e1a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.848889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727404f-e111-460f-84ad-d3ea419f9e1a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.849096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727404f-e111-460f-84ad-d3ea419f9e1a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.850740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 440/840] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:15.856963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.857485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.857762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.866944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94755979-8972-405c-bcd8-2d70d35e8d31 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:15.871286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:15.877433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:15.878614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:15.914414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:15.915408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:15.915806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:15.916578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:15.917620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:15.917934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:15.919529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:15.945266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 401/841] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:15.947018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:15.971880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 441/842] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:15.986676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.047538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.065012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8eeb-4ab8-46b6-81ea-701d615e3152 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.100342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.101239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.101680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.102108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.109891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67e393-3644-4d9a-9e98-7f79728bc90b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.136146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.154428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.155890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.156527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.156900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.171126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:16.179683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:16.181107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:16.182077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:24:16.182909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:16.184434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496055-a888-45f3-a2a9-e64ae055e859 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.187749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 442/843] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:16.188636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.302000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3844-2630-407e-880d-49da796d3ea7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.316638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.342976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.343642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.343957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.344235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.344983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:16.345293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.346661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e047a15-97b4-4746-878a-e7fd6b391d85 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.359212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 443/844] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:16.359212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a0e9-1539-4551-a957-3f46faa0ed4b None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.359212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a0e9-1539-4551-a957-3f46faa0ed4b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.359212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19a0e9-1539-4551-a957-3f46faa0ed4b None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.362009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 444/845] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:16.362660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19a0e9-1539-4551-a957-3f46faa0ed4b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.423567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 402/846] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:16.424115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.424397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.424671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.441758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:16.443034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:16.474584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19a0e9-1539-4551-a957-3f46faa0ed4b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.502369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.511235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.512064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.512347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.512621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.516069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 403/847] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:16.517074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.535131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.537417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:16.555263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:16.556238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:16.557124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:16.557124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:16.557124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:16.557124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:16.560126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 445/848] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:16.560735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.591500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.622506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949439c2-1ec9-4ecb-83f7-8eabf281f502 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.659315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.660145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.660428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.660709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.671237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.672701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.674056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5bf6-f4f8-4b60-b44b-8630e90b028f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:16.691049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:16.692015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:16.692930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:16.692930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:16.692930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:16.692930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:16.692930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 404/849] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:16.693213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.714939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.716032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.716500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.716950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.717795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.719364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:16.731458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:16.732484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:16.733342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:16.740489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 446/850] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:16.741468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.743222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.805839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0b324-7e38-4c12-8c90-b21163a35e55 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.836336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.847372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ad108-e66e-48f0-b60b-486930d454ec None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.849191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.849861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.850139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.850441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.851139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:16.851443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.852870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.865902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 405/851] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:16.866985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.887476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.888380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.888663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.888938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.889655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:16.890024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.891810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.906182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 447/852] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:16.907110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.968527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1d08a-160e-4388-a883-276f7b8b7207 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.970798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.003101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.003751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.004055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.004324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.005018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.005319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.006274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f975526-8c65-431b-b539-3dfdf45388fe None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.006668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5417198-4df8-4209-8774-79f3bf360cba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.011190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 406/853] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.012210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a500bd-53c1-4e41-bc25-c45d0a37bfc6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.012490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a500bd-53c1-4e41-bc25-c45d0a37bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.012763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a500bd-53c1-4e41-bc25-c45d0a37bfc6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.013980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 407/854] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:17.014656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a500bd-53c1-4e41-bc25-c45d0a37bfc6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.042349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.043250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.043601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.043979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.051000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.052463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:17.067791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:17.068906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 448/855] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:17.070190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.081120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.117681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a500bd-53c1-4e41-bc25-c45d0a37bfc6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.152181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.152900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.153178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.153444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.159948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.161363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.169008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73a6fe-9868-4ded-bfa4-da1c42ef5485 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-40b36b9a-b8a4-4b37-af06-a6978cd955d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:17.175540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:17.176458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:17.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 408/856] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:17.178261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.178366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.178563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.194716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:17.195910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:17.206606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.207269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.207549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.207849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.208555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.208862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.210216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.215513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 449/857] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.216327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.219680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.271508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 409/858] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:17.272685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.325937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.354552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61724-686d-4b85-bb88-32263afa72fa None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.396645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.397419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.397709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.397986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.398834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.399146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.400991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81979b-b093-4d90-a95a-c4442ce4e76d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.401334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.423258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 450/859] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.423258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.446779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.447453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.447737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.448059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.448849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.449156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.450583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.463367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 410/860] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.464032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.487515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.539124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb84f2-3b6b-469f-9332-4aab7fc3589a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.571027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.577793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.578789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.579072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.579339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.580024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.581392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.583160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2da0-08af-4b6e-a1b6-339375d6ac3d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:17.593865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:17.594762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:17.595666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 451/861] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:17.597018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.624387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.625043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.625324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.625594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.698998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.700391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.701032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba292a-4a84-40b9-987b-6f360d8d907c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.721076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.736441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.737098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.737380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.737651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.738479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=6bd7c157dada45049506a091a97079bf) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.738793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.740260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.755791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 452/862] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/role_assignments?user.id=6bd7c157dada45049506a091a97079bf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.755791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.825428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 411/863] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:17.831940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.845355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.896840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63a7ee-d939-4b75-86f0-5ecebcf4cc5d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.948242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.949710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.950204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.950913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.951841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:17.952546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.954359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.962633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 453/864] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:17.967257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.978930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4423-4761-4911-bd1f-ca23bc7c2005 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.003244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:18.023351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.024342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.024925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.025295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.037775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.039549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:18.058566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:18.059500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:18.060410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:18.060410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:18.060410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:18.060410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:18.060410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 412/865] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:18.065468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:18.074929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc536-c48a-4b12-a1bd-6ba2ee3bb340 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.098585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:18.111727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.112608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.113162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.113505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.114293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:18.114673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.116173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.122883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 454/866] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:18.194418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d975cd2-da68-4b2b-bdfd-51454d7eff85 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.233264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.233857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.234136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.234405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.235135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:18.235439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.236851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.250587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 413/867] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:18.256909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:18.369031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22da5-c252-48ad-8b0d-fd139db22e98 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.407956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.408752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.409031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.409303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.416773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.418262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:18.436822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:18.437704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:18.438590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 455/868] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:18.444881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:18.476329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:18.548867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c7983-0fa6-49ab-9347-11dd0ff637d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.586505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.587227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.588028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.588297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.589082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:18.589386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.590805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f5f15-a519-44b0-b950-8a1955512058 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.603889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 414/869] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:18.612851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:18.665258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:18.719459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc411-8510-4997-88e6-939141b18b84 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:18.753569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88283564-2855-4426-8b11-73deae3096cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:18.754220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88283564-2855-4426-8b11-73deae3096cd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.754500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88283564-2855-4426-8b11-73deae3096cd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.754775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88283564-2855-4426-8b11-73deae3096cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.755603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88283564-2855-4426-8b11-73deae3096cd None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=bd93f75ca8ee4da0b1f1d89ab59b1733) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:18.755940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88283564-2855-4426-8b11-73deae3096cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:18.757347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88283564-2855-4426-8b11-73deae3096cd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:18.764196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 456/870] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:18 2022] GET /identity/v3/role_assignments?user.id=bd93f75ca8ee4da0b1f1d89ab59b1733&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:19.294190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e2274-d7bd-4560-adc4-ea81bbb6d341 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.295268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e2274-d7bd-4560-adc4-ea81bbb6d341 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.295543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e2274-d7bd-4560-adc4-ea81bbb6d341 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.297295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 415/871] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:19.303195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671501d-bd5f-4198-a798-69a41623a5a4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.303628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671501d-bd5f-4198-a798-69a41623a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.303825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671501d-bd5f-4198-a798-69a41623a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.323153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7671501d-bd5f-4198-a798-69a41623a5a4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:19.324359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7671501d-bd5f-4198-a798-69a41623a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.393433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 457/872] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:19.416761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e2274-d7bd-4560-adc4-ea81bbb6d341 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:19.515077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:19.526388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e2274-d7bd-4560-adc4-ea81bbb6d341 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:19.560555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:19.561135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.561414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.561688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.562416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:19.563857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096246a-a8f2-4066-82d1-4dd342299277 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:19.578244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 416/873] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:19 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:19.998057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62ddb6-b896-4c62-8048-23f17cfefe66 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.998437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62ddb6-b896-4c62-8048-23f17cfefe66 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.998656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62ddb6-b896-4c62-8048-23f17cfefe66 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.000290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 458/874] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:20.005584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105f956-1b44-45f8-aecc-f5ec1a299fd4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.005883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105f956-1b44-45f8-aecc-f5ec1a299fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.006167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105f956-1b44-45f8-aecc-f5ec1a299fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.023755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3105f956-1b44-45f8-aecc-f5ec1a299fd4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:20.025566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3105f956-1b44-45f8-aecc-f5ec1a299fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:20.101031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 417/875] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:20.123226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62ddb6-b896-4c62-8048-23f17cfefe66 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:20.173570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:20.231236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62ddb6-b896-4c62-8048-23f17cfefe66 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:20.266553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:20.267273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.267550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.267844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.268496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:20.269837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:20.282305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:20.283292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:20.284385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 459/876] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:20.287236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332f72-a2dc-4e09-8f22-2f321be14c1c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.287525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332f72-a2dc-4e09-8f22-2f321be14c1c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.291815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92332f72-a2dc-4e09-8f22-2f321be14c1c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.291815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 418/877] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:20.293127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:20.294307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.294732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.294865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.313861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:20.315416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:20.400020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 419/878] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:20.407573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017021d-a0b8-4eea-a1d2-08590899b1e3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:20.414734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:20.417526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:20.438354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:20.447171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:20.447966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.448247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.448523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.449227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:20.449571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:20.450986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:20.456354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 460/879] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:20.461520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:20.547707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7742b9-1048-4e0e-a307-028256649cde None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:20.588358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:20.589029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.589310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.589584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.590133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234723e-3b9f-4109-bdd1-7ae03617625d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:20.590310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:20.591796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131df97-0603-411b-9ee6-fc36be3fcf8b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:20.616353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 420/880] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:20 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:20.634712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:20.635401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.635680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.635983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.636711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:20.637028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:20.638452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0abdf-d2c7-451c-a7ea-4085ed36ab93 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:20.644198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 461/881] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:20.943360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c0be6-1c58-400e-9542-4449a8783379 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.943789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c0be6-1c58-400e-9542-4449a8783379 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.943976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c0be6-1c58-400e-9542-4449a8783379 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.945544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 421/882] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:20.951374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:20.951661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:20.951975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:20.969714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:20.970881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:20.995447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.041083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 462/883] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:21.064277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c0be6-1c58-400e-9542-4449a8783379 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.136798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.179575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c0be6-1c58-400e-9542-4449a8783379 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.216069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:21.217139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.217569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.217985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.228848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:21.237214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:21.238312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:21.239345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:21.241704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 422/884] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:21.248966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.349761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c73b17-165d-4c41-a47f-781926560de1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.386348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:21.387025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.387301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.387576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.388356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:21.388662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:21.390045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:21.399150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 463/885] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:21.404663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.520607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af472a-f7e9-4026-9b3e-130fcd4e4801 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.547739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.558819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:21.559610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.559914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.560188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.571287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:21.572825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-85e36f51-f242-4625-a6f3-c656a3978c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:21.590661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:21.591573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:21.592476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:21.592476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:21.592476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:21.592476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:21.595885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 423/886] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:21.597991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.627014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.709327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925e77f-f0db-418c-83e4-234c799075dd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.736365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216b5b-e37e-46ab-b985-a5e4bc2e41ad None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.736696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216b5b-e37e-46ab-b985-a5e4bc2e41ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.736925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216b5b-e37e-46ab-b985-a5e4bc2e41ad None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.738511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 424/887] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:21.744144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.744425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.744696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.745566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:21.746346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.746629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.746905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.747635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:21.747974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:21.749351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:21.761690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:21.761861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 464/888] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:21.765042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:21.767383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.847331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 425/889] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:21.867303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.872017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.890602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c74b-56d0-41f5-96bf-8866ab9c8052 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.927880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:21.928692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.928969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.929241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.939199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:21.941287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:21.960322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:21.961270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:21.962735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:21.966958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 465/890] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:21.967798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:21.985430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68159d0d-31c4-4e87-93b3-e63772dc79cc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:21.995924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.041863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.042939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.043227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.043506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.044199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.045564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:22.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:22.062112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:22.063191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 426/891] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:22.069164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.086321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ae8c-0358-4c13-a442-13c06bb5f076 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.125118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.125709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.125988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.126268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.127113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:22.127425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.129145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.144348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.150004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 466/892] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:22.155432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.208925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb105e-2939-4cba-83bb-7b78679fe892 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.253148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.254288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.254786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.255281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.256306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:22.256881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.258469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.263962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 427/893] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:22.269358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.271460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1437-3824-4a31-937e-dea9f56dfd2c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.308041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.321598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.323057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.323794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.324276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.390837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f265-be6c-4db6-a841-b658bce7f238 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.413536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.418361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.420476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.441801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.442483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.442769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.443047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.443794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:22.444220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.445706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.456292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 428/894] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:22.458211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.544514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 467/895] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:22.545626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.587470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.591002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3361c-b245-4373-93e7-e28cb2550ec7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.639963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.641426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.641855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.642266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.643678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] RBAC: Adding query filter params (interface=public, service_id=a47ad27c8d944d8c90127c11501ab9cf, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:22.644167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.646395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.659161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 429/896] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/endpoints?service_id=a47ad27c8d944d8c90127c11501ab9cf&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:22.659557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.675462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.680628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3bbb-0f41-45dd-9ffd-9f2975b3adc0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.726342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.727724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.728906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.729352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.743073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.745329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-4e068936-2f53-42fa-921b-fa87d0e08bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:22.765250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:22.766356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:22.767491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:22.767491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:22.767491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:22.767491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:22.769031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 468/897] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:22.770505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7248-a032-42d1-8a4b-173335e26078 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.770793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7248-a032-42d1-8a4b-173335e26078 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.771066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd7248-a032-42d1-8a4b-173335e26078 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.772569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 469/898] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:22.774374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420f13f-028c-49ae-978d-28aa339c778d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.775084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd7248-a032-42d1-8a4b-173335e26078 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.816986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.817652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.818351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.818628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.819311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.821071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32569a-33cc-4148-8368-39fcb940d6ba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.828344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 430/899] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:24:22.829489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34cb60-0539-443f-9973-3c7abc73b10d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.829765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34cb60-0539-443f-9973-3c7abc73b10d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.830031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34cb60-0539-443f-9973-3c7abc73b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.848382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4d34cb60-0539-443f-9973-3c7abc73b10d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:22.849602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d34cb60-0539-443f-9973-3c7abc73b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:22.876381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.887244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd7248-a032-42d1-8a4b-173335e26078 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.927563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16495d05-360e-4684-a91b-727b50214075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.928176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16495d05-360e-4684-a91b-727b50214075 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.928463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16495d05-360e-4684-a91b-727b50214075 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.928744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16495d05-360e-4684-a91b-727b50214075 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.940014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 431/900] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:22.940218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16495d05-360e-4684-a91b-727b50214075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:22.940218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16495d05-360e-4684-a91b-727b50214075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.940387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8aea9-0421-47a4-aba2-0abdc8a66e61 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.940387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8aea9-0421-47a4-aba2-0abdc8a66e61 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.940387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8aea9-0421-47a4-aba2-0abdc8a66e61 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.940642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16495d05-360e-4684-a91b-727b50214075 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.940726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 432/901] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:22.940726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.940726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.940726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.953221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 470/902] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:22.958679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16495d05-360e-4684-a91b-727b50214075 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.958858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:22.959971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:22.976337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.037979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 433/903] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:23.039356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.074515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16495d05-360e-4684-a91b-727b50214075 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.109185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.110103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.110456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.118942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.120524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:23.135821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:23.136769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:23.137648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 471/904] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:23.138130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.145691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.159311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade40-3f7c-4a14-b6fe-7053978bb9c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.196848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.197689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.197967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.198350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.199001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.200477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.209572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:23.213940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:23.214896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:23.216248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 434/905] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:23.217358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.245314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042db0f-dee2-4c1f-8e65-c2a40157cba5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.282901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.283731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.284045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.284327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.285053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.286427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.301359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 472/906] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:23 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:23.302100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.359483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918edd6f-6917-4c4c-bf72-2898207f28e6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.410838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.411509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.411825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.412104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.412997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:23.413308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.414738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.429419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 435/907] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:23.437128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.442736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9fae7-5431-4def-a8bf-68c911f41d40 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.466126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.476273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.483702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.484433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.484726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.485013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.485741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:23.486054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.487557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.493633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 473/908] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:23.500256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.562905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e41dd-d113-4790-a443-18bef349b70f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.600823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.601526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.601885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.602234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.603838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] RBAC: Adding query filter params (role.id=33453989f041479e8edd4baf6eed27c3, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=bd93f75ca8ee4da0b1f1d89ab59b1733) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:23.604232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.605962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.620078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 436/909] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/role_assignments?user.id=bd93f75ca8ee4da0b1f1d89ab59b1733&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=33453989f041479e8edd4baf6eed27c3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:23.622200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fca68-9406-4149-ac3c-a58be35fbfe1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.660945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.661886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.662314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.662761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.663633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:23.664115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.666248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.672693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 474/910] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:23.681380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.756784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.788316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2c829-d307-4008-878a-8f73abf53d76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.832984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.833656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.833942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.834220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.835015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] RBAC: Adding query filter params (interface=public, service_id=fb62baa8c602434187578b8d0431e37e, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:23.835328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.836776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bc640-4d2f-4cf1-a543-c56cc87b174a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:23.842790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 437/911] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/endpoints?service_id=fb62baa8c602434187578b8d0431e37e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:23.847997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:23.909126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.952671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7b9-fb6d-49ef-8adb-cc578307464e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:23.989113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:23.990150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:23.990673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:23.991141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:23.992048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:23.994421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962fba4-ae2b-4e4a-9824-b171824aa7c3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:24.001305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 475/912] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:23 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:24:24.900413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea593f0-3b4f-4002-927e-9a2e0d944df7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:24.900986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea593f0-3b4f-4002-927e-9a2e0d944df7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:24.900986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea593f0-3b4f-4002-927e-9a2e0d944df7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:24.902549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 438/913] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:24.907832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:24.908244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:24.908435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:24.925521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:24.926700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:24.994612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 476/914] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:25.018021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea593f0-3b4f-4002-927e-9a2e0d944df7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.126373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea593f0-3b4f-4002-927e-9a2e0d944df7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.163043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.163795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.164194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.164478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.165201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.166564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:25.178671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:25.179748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.181204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 439/915] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:25.187885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.241964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.289185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106628a-79e8-4d45-9d08-ceed3789b9b9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.324210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.325233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.325721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.326199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.327116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:25.327629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.329199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.334526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 477/916] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:25.341125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.453077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a1bab-f673-42f6-b0b9-e8cbc66ffc60 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.487514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.488209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.488492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.488756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.489463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:25.489767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.491130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932b149-a62b-4217-a62a-b924fc237165 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.497138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 440/917] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:25.502694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.536002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524deb3-5379-40e8-804a-02ce15c789c2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.536588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524deb3-5379-40e8-804a-02ce15c789c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.537052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524deb3-5379-40e8-804a-02ce15c789c2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.539095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 441/918] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:25.544432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.544975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.545443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.563498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:25.565030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:25.607851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24edf0-f725-4378-9281-716c907ab25e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.630453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.635496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.643778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.644706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.645181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.645654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.646599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.648199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf111d7d-f703-4cf8-bc5c-c4414cd7d4c0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.655842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 442/919] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:25.673117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 478/920] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:25.675177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.745020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e54f2-8fcf-46f1-a1ce-322db301a193 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.745888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e54f2-8fcf-46f1-a1ce-322db301a193 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.746395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e54f2-8fcf-46f1-a1ce-322db301a193 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.749132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 479/921] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:25.755088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.755738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.756280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.775141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:25.776761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:25.796260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9fbc1-906a-4623-bbc7-bb3cfa6bf319 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.844415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.845596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.846095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.846580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.847437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.849040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.852276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 480/922] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:25.869481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:25.870614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:25.871774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 443/923] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:25.878111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.883042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.957282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.988836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1323c6-f4ab-410b-a39c-c6c0c8f5b927 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.992182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.024489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894a761-9a0c-4a1f-9fa8-4d3df9bd6114 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.033543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.034441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.035228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.035673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.036921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.037714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.039478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.051079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 481/924] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:26.052206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.068412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.069966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.070641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.071064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.085905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:26.097379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:26.098538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:26.099579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:26.102162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 444/925] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:26.107935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.169508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca74ac1-0887-4aef-a310-a71eac001aec None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.203433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.204164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.204447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.204725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.205440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.205749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.207151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6570c4-2560-462f-a7b6-3f8dabeb43a9 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.215538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 482/926] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:26.217075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11393e-faaa-4dda-8408-3672a4e209ad None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.217450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11393e-faaa-4dda-8408-3672a4e209ad None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.217798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11393e-faaa-4dda-8408-3672a4e209ad None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.219106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 483/927] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:26.224036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11393e-faaa-4dda-8408-3672a4e209ad None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.232042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd07be6-008c-436a-a8c7-ba3f5d7e29eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.245854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.258648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.269972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.270647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.270929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.271201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.271986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.272292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.273665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26196e67-50e8-4edb-8fdc-d1ff940dee66 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.287120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 445/928] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:26.287120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.287120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.287120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.300399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:26.301585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:26.343704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11393e-faaa-4dda-8408-3672a4e209ad None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.366858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 446/929] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:26.367694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.392112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.392785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.393273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.393820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.394906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] RBAC: Adding query filter params (interface=public, service_id=48afb1b59ff44a8eb5ff0b19de237187, region_id=RegionOne) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.395227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.400598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.404361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 484/930] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/endpoints?service_id=48afb1b59ff44a8eb5ff0b19de237187&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:26.405024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.497572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01336bcc-1195-4fac-85bd-be4336724b27 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.506849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.517528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.526015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a487c71-fb0a-402f-bd5e-b7ff34b834b2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.538692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.539715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.540734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.541171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.551641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.553324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.561082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.561881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.562158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.562602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.563337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.564609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:26.571571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:26.572484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:26.573374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:26.573374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:26.573374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:26.573374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:26.576098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 447/931] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:26.577012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:26.578207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:24:26.579201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:26.580165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.581189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 485/932] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:26.581957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.692328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33179-b3ce-44b7-bd08-2688616747fd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.695738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21f73-6634-4a97-8dd0-a687a6eb7680 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.729062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.729729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.730011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.730283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.731007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.731316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.732702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.732803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.733462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.733747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.734038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.734714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.736092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.752198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 448/933] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:24:26.752198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.757298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 486/934] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:26.764688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.783476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.788009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:26.879844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ab00-fa62-47c6-9e4e-1a0067f48162 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.905557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6422ba-b4e0-4d7a-bbf7-f5cced3c233e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:26.928034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.928872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.929162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.929422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.939343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.940836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.947336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:26.948100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:26.948388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:26.948666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:26.949376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:26.949682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:26.951033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:26.956781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:26.957727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:26.958693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:26.959263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 449/935] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:26.963111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 487/936] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:26.964457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:26.969746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.057226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.083174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.085802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0179-e75a-47b0-bdd2-09f8b8a85b24 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.102085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93ded0-4942-4e04-908e-8b7cc0cc33f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.134472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.135070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.135353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.135627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.136505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:27.136819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.138248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.155895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.156567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.156848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.157120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.157867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:27.158182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.159532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.166203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 488/937] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:27.171806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 450/938] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:27.172740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.173211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.286137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bb83a-e142-412f-a2a0-2e282af3078d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.318568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a7aa6-9780-465f-af9d-f5b17aaab827 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.324158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.325043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.325471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.325813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.332898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.339042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.340634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:27.359729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:27.360673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:27.361731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:27.361731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:27.361731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:27.361731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:27.361731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 489/939] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:27.364399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.365215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.365719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.366128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.366622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.367167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.368975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.390530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 451/940] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:27.495158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfd185-7887-41d7-b75e-fec9fa3b9f5b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.540393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.541077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.541415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.541699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.542450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:27.542789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.544221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.557495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 490/941] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:27.563190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.574734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.680826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b900a-8d54-4239-ace3-034273b9524b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.720350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.721156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.721439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.721716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.728919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.730370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1182bf59-30fd-4945-a288-6354ba3a9a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:27.745861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:27.746784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:27.748106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 452/942] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:27.753059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.802574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde9f9-677c-422f-af44-b967af52505c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.802912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde9f9-677c-422f-af44-b967af52505c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.803125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcde9f9-677c-422f-af44-b967af52505c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.804946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 453/943] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:27.806489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.810719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.811004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.811278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.834356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:27.835576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:27.865754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7df1-d3c8-44a5-83ad-cbe9d0cb475a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:27.901346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:27.902377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:27.902933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:27.903404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:27.904418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:27.904953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:27.906655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:27.914123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:27.920649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 491/944] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:27.926719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:27.936040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 454/945] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:27.957726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:28.058598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef4e31-527b-4f2b-b38e-10642235fe5a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:28.065051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47df42-256f-47ee-86f9-7c25fd2bde81 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:28.069782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:28.101940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:28.102629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.102907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.103177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.103909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:28.105328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:28.106885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING keystone.server.flask.common [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 15:24:28.107701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:28.109010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.109506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.110008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/bd93f75ca8ee4da0b1f1d89ab59b1733/roles/33453989f041479e8edd4baf6eed27c3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.113122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=bd93f75ca8ee4da0b1f1d89ab59b1733, role_id=33453989f041479e8edd4baf6eed27c3)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:28.114766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:28.125460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.sql.core [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:24:28.125460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 15:24:28.125460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [parameters: {'id': '994947fcebe3496bae66846ae0552c45', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 15:24:28.125460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80207) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 994947fcebe3496bae66846ae0552c45.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 994947fcebe3496bae66846ae0552c45. Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:24:28.139824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 15:24:28.140715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application yield resource Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise self.value Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application session.commit() Jul 20 15:24:28.141670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:24:28.142594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:24:28.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:24:28.145801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [parameters: {'id': '994947fcebe3496bae66846ae0552c45', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:28.149227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:28.151233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 994947fcebe3496bae66846ae0552c45. Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 455/947] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Jul 20 15:24:27 2022] POST /identity/v3/projects => generated 262 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:28.153418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:28.155887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 492/947] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:27 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/bd93f75ca8ee4da0b1f1d89ab59b1733/roles/33453989f041479e8edd4baf6eed27c3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:24:28.218330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:28.275260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bb30-b28f-4724-b62c-ce1d07db2a92 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:28.313651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:28.314933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.315355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.315815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.323841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:28.325755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cd6396bd-15ea-44aa-bc3e-2d8b282b43b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:28.341099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:28.342172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:28.343251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:28.345364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 456/948] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:28.353466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:28.459068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c9f97-130d-4459-b7b4-96824ad06fa1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:28.495478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:28.496234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.496639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.497009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.497863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:28.498265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:28.499798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244201-9e08-402b-a7ac-c821e7294e7b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:28.517966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 493/949] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:28.933105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c14eb-493d-4925-a975-3fbb8a7fa2c5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.933484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c14eb-493d-4925-a975-3fbb8a7fa2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.933687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c14eb-493d-4925-a975-3fbb8a7fa2c5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.938082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 457/950] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:28.943363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:28.943657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:28.943967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:28.961506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:28.962780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:28.983980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.036277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 494/951] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:29.059891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c14eb-493d-4925-a975-3fbb8a7fa2c5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:29.165821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c14eb-493d-4925-a975-3fbb8a7fa2c5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:29.179065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.201955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:29.202691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.202975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.203247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.203939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:29.205372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:29.217740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:29.218960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:29.220085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 458/952] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:29.229057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:29.337647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c16c9-474d-4a76-8e8e-d3308ac921bd None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:29.374290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251235a-2670-4f49-b263-373818a9132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:29.374959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251235a-2670-4f49-b263-373818a9132e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.375279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251235a-2670-4f49-b263-373818a9132e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.375581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251235a-2670-4f49-b263-373818a9132e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.376335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251235a-2670-4f49-b263-373818a9132e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:29.376648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251235a-2670-4f49-b263-373818a9132e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:29.378033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251235a-2670-4f49-b263-373818a9132e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:29.383888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 495/953] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:29.389610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:29.499028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79cd0-a31a-491d-a3c9-99cb14b07281 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:29.535002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:29.535675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.535989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.536268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.536974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:29.537282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:29.538770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e620bc-7702-48c3-9ef0-63f1a03939e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:29.544711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 459/954] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:29.550356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251235a-2670-4f49-b263-373818a9132e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:29.572608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.654524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251235a-2670-4f49-b263-373818a9132e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:29.671699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5897e-c1b5-4af7-80ef-8613546bce9f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.672126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5897e-c1b5-4af7-80ef-8613546bce9f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.672317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5897e-c1b5-4af7-80ef-8613546bce9f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.673901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 460/955] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:29.680022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.680437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.680859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.696431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:29.697099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.697386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.697700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.698411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:29.699936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:29.700466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:29.702185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:29.731972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 496/956] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:29.761442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.799221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 461/957] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:29.823022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:29.913323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.939225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730262b-2804-46a1-94cc-27cd3b0dee1f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:29.975813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:29.976577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:29.976864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:29.977138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:29.977824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:29.979188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:29.991184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:29.992291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:29.993839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 497/958] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:29.998904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.107191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b631d3a-7734-4ce6-ac46-172918ecf579 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.144858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:30.145838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.146411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.146866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.147752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:30.148272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:30.149911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db879ea-5ffd-46e9-947e-e949a7e63777 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:30.155495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 462/959] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:30.161468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.260352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b748-707f-437a-896c-3cf497a73125 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.260886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b748-707f-437a-896c-3cf497a73125 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.260886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b748-707f-437a-896c-3cf497a73125 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.262643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 463/960] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:30.267833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.268131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.268519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.274875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c530ef-0d73-47a8-87c3-750e1814006a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.286204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:30.287715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:30.313125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:30.313798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.314080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.314359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.315062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:30.315373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:30.316805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:30.319984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:30.324090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 498/961] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:30.375334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 464/962] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:30.400143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.405563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:30.532208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea54-12e5-4bf4-866f-b4fd0dc1b8a5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.568814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:30.569638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.569921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.570202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.581229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-90e3ba6c-b22c-4b28-8b67-baa39f6502ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:30.589627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:30.590569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:30.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 499/963] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:30.596780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.703796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c56b7-7d8f-4a97-8f59-f0ef5616c1d7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.707134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf222-d924-4f0d-b75a-63e106ae50ef None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.707425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf222-d924-4f0d-b75a-63e106ae50ef None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.707711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caf222-d924-4f0d-b75a-63e106ae50ef None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.709362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 500/964] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:30.714306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.714594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.714877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.738390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:30.739600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:30.748673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:30.749346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.749983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.750282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.751038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:30.751660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:30.753361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:30.764308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 465/965] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:30.770084 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.802098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:30.817701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:30.831924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 501/966] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:30.859241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:30.904581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734a097-155c-41e6-90d6-2fd58b8b5d2c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.955221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:30.956417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:30.956908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:30.957984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:30.971586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496948b-7715-4e62-b4d5-abeb795f89ff None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:30.973222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:30.975691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:30.999111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:31.000126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:31.000998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:31.000998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:31.000998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:31.000998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:31.002060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 466/967] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:31.010899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.016289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.017050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.017329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.017605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.018382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.019934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.021447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING keystone.server.flask.common [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 15:24:31.035543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.sql.core [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:24:31.035543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 15:24:31.035543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [parameters: {'user_id': '9250badfec69485ebb079a652ce1e440', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 15:24:31.035543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80208) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application result.read() Jul 20 15:24:31.054562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:24:31.055707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application yield resource Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise self.value Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application session.commit() Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:24:31.057002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:24:31.058131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:24:31.059216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application result.read() Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:24:31.065679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [parameters: {'user_id': '9250badfec69485ebb079a652ce1e440', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:31.067635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 15:24:31.070150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 502/968] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jul 20 15:24:30 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:31.072696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.116667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:31.151011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:31.151509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e056-dc4c-4d01-bf09-39ce28f70b44 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.193044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.193704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.193982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.194252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.194980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:31.195267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.197135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.213600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 467/969] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:31.214726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301687-d69f-4c41-b730-fabdd773df11 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.219903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.250656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.251385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.251721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.251960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.260599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.262215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:31.277406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:31.278927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:31.279931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 503/970] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:31.284370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.339280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483002b-14a7-486b-8335-a5c047239a8f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.382039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.382853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.383131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.383402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.386310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:31.394896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.396528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.414190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61363-4c72-4584-a9e0-24d6375ed07e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:31.416316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:31.417297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:24:31.418323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:31.427851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 468/971] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:31.433242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.444040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:31.469570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.470736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.471182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.471790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.472815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:31.473594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.475332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.526731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 504/972] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:31.564406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b794a4-7ffe-4f57-94cb-565a7aab7dd0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.605355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.606046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.606323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.606593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.607516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:31.607867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.609474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d6421-93ca-4b13-98de-c649dd973db4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.637290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 469/973] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:31.639010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.746529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e5710-5f06-467f-858f-4509ba9bf0cd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:31.781462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:31.782116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.782389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.782661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.827321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:31.855127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:31.856904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:31.956984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 505/974] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:31.957393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17891dc-03ff-4962-b67e-4a19701fe301 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.957683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17891dc-03ff-4962-b67e-4a19701fe301 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.958028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17891dc-03ff-4962-b67e-4a19701fe301 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.959661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 470/975] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:31.964850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:31.965141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:31.965431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:31.965718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:31.982411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:31.983599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:32.038900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.049366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.063816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 471/976] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:32.080488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972a60-53ea-46a9-80c7-f03441d0f944 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.082930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.116462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.118441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.118721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.118992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.131319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.133019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:32.153328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:32.154373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:32.155259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:32.155259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:32.155259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:32.155259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:32.157195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 506/977] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:32.163061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.194915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca16b-d76f-4c1a-939e-108a1fc638f2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.234113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.234918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.235197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.235464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.236143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.237498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:32.249888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:32.250877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:32.251937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 472/978] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:32.256813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.271978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469217c8-27f1-4e7e-b67d-0d4e7bb05f72 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.299895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.306209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.311601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.312300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.312581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.312906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.313631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.313984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.315397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.332362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 507/979] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:32.333264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.365403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a0631-3964-48c2-8b97-b62a76067724 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.406067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.406833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.407183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.407526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.408501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.408875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.410708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.421311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 473/980] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:32.422674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.449609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501bcec-40bc-442b-a502-540d38cb943a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.492571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.494136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.494411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.494679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.501313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.502739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-4d7bd168-beae-41ee-be8c-50eb24565119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:32.517279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:32.518380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:32.519453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 508/981] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:32.520019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d967-a093-41c6-84c0-ec4b27d542fc None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.520171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d967-a093-41c6-84c0-ec4b27d542fc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.520450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41d967-a093-41c6-84c0-ec4b27d542fc None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.521960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 509/982] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:32.523997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41d967-a093-41c6-84c0-ec4b27d542fc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.534735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e180c1-bc04-4a31-9b2f-ad3a66c7b076 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.544997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.569711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.570313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.570590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.570866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.572147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.572376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.572473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.573914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867c19-c7f8-4592-80f0-721e45887ddb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.578629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 474/983] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:32.579801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.580087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.580362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.596150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:32.597313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:32.631379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41d967-a093-41c6-84c0-ec4b27d542fc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.662626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 475/984] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:32.663437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.664912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.665564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.665844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.666119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.666903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.667209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.668664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.683364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 510/985] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:32.685652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.767395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083579ea-3b55-4b20-876d-f519a625b2ce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.769643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.784889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9edb6-f8bf-4c81-9910-b6c1472ca970 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.792402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:32.802869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.803523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.803834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.804116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.804909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] RBAC: Adding query filter params (interface=admin, service_id=48afb1b59ff44a8eb5ff0b19de237187, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.805218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.807587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.812476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 476/986] 10.222.0.27 () {56 vars in 1249 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/endpoints?service_id=48afb1b59ff44a8eb5ff0b19de237187&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:32.813345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.822707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.823292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.823573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.823881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.824599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.825952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.839112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 511/987] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:32 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:32.840144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:32.931357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04c5f-457a-470f-9696-30586d84b3e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.962615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda48e18-1d8c-43d4-b048-46a139622510 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:32.970907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:32.971590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:32.971966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:32.972243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:32.973119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] RBAC: Adding query filter params (role.id=33453989f041479e8edd4baf6eed27c3, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=bd93f75ca8ee4da0b1f1d89ab59b1733) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:32.973434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:32.974894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d47b29-0f67-43c8-a72f-de566233d790 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:32.982278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 477/988] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/role_assignments?user.id=bd93f75ca8ee4da0b1f1d89ab59b1733&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=33453989f041479e8edd4baf6eed27c3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:33.008259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:33.008950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:33.009239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:33.009516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:33.010222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:33.011656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d52d36-ecf2-46d6-8bc5-fe20026a304e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:33.019990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 512/989] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:32 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:24:34.934546 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431fd-5331-4a0b-8b3c-77da437a9e1c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:34.935066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431fd-5331-4a0b-8b3c-77da437a9e1c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:34.935142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f431fd-5331-4a0b-8b3c-77da437a9e1c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:34.936661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 478/990] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:34.941715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cfa6d-632e-4832-be54-2f3921ca21ca None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:34.942098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cfa6d-632e-4832-be54-2f3921ca21ca None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:34.942276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cfa6d-632e-4832-be54-2f3921ca21ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:34.959306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8d9cfa6d-632e-4832-be54-2f3921ca21ca None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:34.960831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8d9cfa6d-632e-4832-be54-2f3921ca21ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:34.982747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.032830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 513/991] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:35.054853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f431fd-5331-4a0b-8b3c-77da437a9e1c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.074297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee14884-466d-4702-9471-b4fe2d411fa2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.074587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee14884-466d-4702-9471-b4fe2d411fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.074868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee14884-466d-4702-9471-b4fe2d411fa2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.076412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 514/992] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:35.081140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283e57a-9a2a-4fe6-b64c-8229edb045cc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.081424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283e57a-9a2a-4fe6-b64c-8229edb045cc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.081706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283e57a-9a2a-4fe6-b64c-8229edb045cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.088832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.100519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8283e57a-9a2a-4fe6-b64c-8229edb045cc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:35.101699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8283e57a-9a2a-4fe6-b64c-8229edb045cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:35.161230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f431fd-5331-4a0b-8b3c-77da437a9e1c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.170245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 515/993] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:35.171552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c46f-1445-4dbc-b20e-cb11db9cee03 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.171871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c46f-1445-4dbc-b20e-cb11db9cee03 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.172150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c46f-1445-4dbc-b20e-cb11db9cee03 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.173995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 516/994] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:35.178754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.179034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.179371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.196801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:35.197277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.197893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:35.198073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.198358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.198632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.199286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.200678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:35.212773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:35.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.215242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 479/995] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:35.216078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.266769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 517/996] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:35.267595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.284353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.323735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.330769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae74d-32ca-4250-a6c5-df4fc15954d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.364076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93258705-5253-48de-9458-eebf63bb876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.364858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258705-5253-48de-9458-eebf63bb876e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.365133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258705-5253-48de-9458-eebf63bb876e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.365400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258705-5253-48de-9458-eebf63bb876e None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.366040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93258705-5253-48de-9458-eebf63bb876e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.367394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93258705-5253-48de-9458-eebf63bb876e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.370317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301d0a-adb3-4a3e-b5ab-ae73cf3272ba None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-93258705-5253-48de-9458-eebf63bb876e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:35.378982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:35.380011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.380867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 480/997] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:35.381673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93258705-5253-48de-9458-eebf63bb876e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.407091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.407798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.408088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.408365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.409075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.409388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.410748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.416077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 518/998] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.416849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.483248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93258705-5253-48de-9458-eebf63bb876e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.500918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.517894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.518814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.519103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.519378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.520058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.520922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca54f2-f0dc-4df0-8ca5-d0031fce18bb None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.521424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:35.533970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:35.535102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:35.536046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 481/999] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:35.536902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.560051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.560716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.561001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.561276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.561371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.562115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.562278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.563866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.568863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 519/1000] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.569640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.635915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c6141-0df8-4cde-bdda-18f85de18ee4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.666031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fe77d-3b75-415f-ac7e-82f85e925be8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.669489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.670148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.670434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.670712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.671418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.671726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.673116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.677992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 482/1001] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.678790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.703713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.704389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.704669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.704940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.705639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.705943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.707322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.712828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 520/1002] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.713592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.723612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.769261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.782051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b336ec1-fb44-47a3-bd40-254a79f0ce0b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.811664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe2dec-905f-4d36-a80e-f8df7ad6d750 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.814430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.815092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.815374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.815654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.816390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.816700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.818052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.822977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 483/1003] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.823739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.857757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.858437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.858742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.858980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.860007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] RBAC: Adding query filter params (interface=public, service_id=542cabaf72224bad92b1b14a39eb21da, region_id=RegionOne) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.860187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.861548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.867510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 521/1004] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/endpoints?service_id=542cabaf72224bad92b1b14a39eb21da&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:35.874130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:35.942906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7ed3-f6fd-4f99-90c4-78a49f1623ac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.953714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.988778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:35.989455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:35.989741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:35.990018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:35.990734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:35.991046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:35.992467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:35.994631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f07b40-2c5b-41da-b870-45f31b2badd4 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:35.999810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 484/1005] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:36.004019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:36.025793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:36.030452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:36.031193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:36.031481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:36.031785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:36.032513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:36.033975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a4f04-ae19-4cff-934d-ca515dc7468e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:36.040586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 522/1006] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:35 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:24:36.121561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b743-aed9-44c9-bfd0-5d805bbbadef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:36.158527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:36.159214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:36.159497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:36.159808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:36.160568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:36.161969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c9566-c32f-4a9b-83ba-5d5fac4bed28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:36.179877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 485/1007] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:37.882782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3c761-3762-482d-8a51-768d47a5f0ee None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:37.883300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3c761-3762-482d-8a51-768d47a5f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:37.883377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3c761-3762-482d-8a51-768d47a5f0ee None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:37.884818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 523/1008] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:37.889876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373cfc4-3597-44b1-8e09-6122866697dc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:37.890285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373cfc4-3597-44b1-8e09-6122866697dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:37.890446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373cfc4-3597-44b1-8e09-6122866697dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:37.907747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c373cfc4-3597-44b1-8e09-6122866697dc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:37.908962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c373cfc4-3597-44b1-8e09-6122866697dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:37.984420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 486/1009] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:38.010211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3c761-3762-482d-8a51-768d47a5f0ee None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.097286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef652fa-dc55-49c1-963b-ff6a1b094a42 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.097625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef652fa-dc55-49c1-963b-ff6a1b094a42 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.097838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef652fa-dc55-49c1-963b-ff6a1b094a42 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.099366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 487/1010] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:38.103993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3fb17-b828-459a-b7c9-2ea9da70d823 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.104277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3fb17-b828-459a-b7c9-2ea9da70d823 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.104599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3fb17-b828-459a-b7c9-2ea9da70d823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.111014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.113168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3c761-3762-482d-8a51-768d47a5f0ee None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.126369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66c3fb17-b828-459a-b7c9-2ea9da70d823 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:38.128300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66c3fb17-b828-459a-b7c9-2ea9da70d823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:38.154259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.154853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.155395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.155395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.156166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.157540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6609fe-7542-40a6-a499-766ec8066c23 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.160513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.187792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 524/1011] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:38 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:38.212368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 488/1012] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:38.226446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd33546-9e34-4075-a51c-6710a0912d26 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.226796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd33546-9e34-4075-a51c-6710a0912d26 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.227018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd33546-9e34-4075-a51c-6710a0912d26 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.228721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 525/1013] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:38.234377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.234705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.234943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.237312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd33546-9e34-4075-a51c-6710a0912d26 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.253057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:38.254276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:38.333349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 489/1014] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:38.352601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd33546-9e34-4075-a51c-6710a0912d26 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.355800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.394776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.395607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.395919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.396198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.396857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.398227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:38.411113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:38.412353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:38.413183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 526/1015] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:38.419005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.431458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.470978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f179d4-c08d-4b3c-8617-d05e6e6c7ea1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.488135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.512808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.513902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.514352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.514783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.516148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.518138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.520288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 490/1016] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:38.526317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.537363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad749c0-21f5-44e1-abb7-0b760b697873 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.575589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.576447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.576733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.577011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.577722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:38.578031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.579560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.590052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 527/1017] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:38.591027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.625986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083921-14c3-4524-8342-f95784d8ae6c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.652680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.661989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.662590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.662870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.663138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.663878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.665323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.692390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 491/1018] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:24:38 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:38.706796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104c46-e409-4185-9f0a-b0798723c01e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.735474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:38.749265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.750018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.750306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.750561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.751329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:38.751638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.753068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f648-d82d-4689-baa0-06622f611a1c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.759645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 528/1019] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jul 20 15:24:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:38.765178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:38.874601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd32193-4bc3-4c22-8a7f-3e7e6ff256b8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:38.909036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:38.909779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:38.909987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:38.910255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:38.911023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:38.912494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43d66d-6efc-42a2-bd05-fb989b1f8d4f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:38.933097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 492/1020] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:40.266128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2fd2-a500-4725-b2fa-6a0c0f813d4e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.266643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2fd2-a500-4725-b2fa-6a0c0f813d4e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.266765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d2fd2-a500-4725-b2fa-6a0c0f813d4e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.268126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 529/1021] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:40.273238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.273623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.273795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.291808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:40.292984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:40.320569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:40.370031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 493/1022] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:40.392037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d2fd2-a500-4725-b2fa-6a0c0f813d4e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:40.500608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d2fd2-a500-4725-b2fa-6a0c0f813d4e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:40.536171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:40.536900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.537184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.537458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] PATH_INFO: `/v3/services/image` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.538109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:40.539465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:40.554196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:40.555543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:40.556407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:40.557778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 530/1023] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jul 20 15:24:40 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:40.561629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:40.664789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fcf67-4614-4566-8767-bc89a9af4e53 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:40.699559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:40.700250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.700530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.700807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.701504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:40.701809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:40.703164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c050728-59f9-4809-869a-075ed54f3e89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:40.713301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 494/1024] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:40.714500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:40.764089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b13a3-17f7-49de-8ae2-294bf908903c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.764380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b13a3-17f7-49de-8ae2-294bf908903c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.764871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b13a3-17f7-49de-8ae2-294bf908903c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.766459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 495/1025] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:40.768963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:40.771532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.771892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.772122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.789510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:40.790641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:40.808948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:40.819513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3661-5fa3-4c46-be23-11ab05bd9684 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:40.855707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:40.856418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:40.856700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:40.856976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:40.857686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] RBAC: Adding query filter params (type=image) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:40.857996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:40.859356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:40.862207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 496/1026] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:40.869425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 531/1027] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:40.870552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:40.886855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:40.975569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e06b5-899b-464a-aea0-cb809de7ee66 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:40.995602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc2a7-a96a-4a4e-8aff-ef4256646a35 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.012557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.013204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.013408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.013649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.013924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.015496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] RBAC: Adding query filter params (interface=public, service_id=e72cccb3c85c4da886d14ee39bd0d961, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:41.015861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.018042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.030351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 497/1028] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:24:40 2022] GET /identity/v3/endpoints?service_id=e72cccb3c85c4da886d14ee39bd0d961&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:41.031387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.032392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.033197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.033478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.033756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.034943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.036414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.038037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 532/1029] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:41.043657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.053890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.135904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79e9e-0f16-4031-9aba-bc489723fc74 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.143227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac38bfc-d231-49b6-82ac-1351fbc079a2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.170452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.171122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.171404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.171683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.172397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.173757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc597f9-cae4-47be-83c0-dbf4b1d0cd79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.177205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.177863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.178144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.178412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.179111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.180210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 498/1030] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:24:41.181114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.181905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db324-d191-49f7-8cca-0f717aafb1a0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.182189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db324-d191-49f7-8cca-0f717aafb1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.182462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db324-d191-49f7-8cca-0f717aafb1a0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.183990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 499/1031] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:41.189539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.189857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.190101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.209137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:41.210679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:41.216752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 533/1032] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:24:41 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:41.296158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 500/1033] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:41.320062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.372226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.443356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b153a-d2e7-48eb-b00d-eb319b062402 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.479777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.480646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.480929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.481301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.491699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:41.500726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:41.501752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:41.502817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 534/1034] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:41.507944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.522434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.609745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa7b21-9a09-4953-9bff-6691b29efc44 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.643869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.644539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.644818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.645087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.645826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:41.646126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.647470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.660249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 501/1035] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:41.663181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.761396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.765208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2032bd2-999d-4128-a457-6d4f2bc6edac None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.799966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.800754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.801034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.801308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.812766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.814207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:41.831829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:41.832919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:41.833935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:41.833935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:41.833935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:41.833935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:41.833935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 535/1036] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:41.839394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:41.915534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:41.941614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fb705-0343-40a1-bffd-55b0d37a15bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:41.975294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:41.975993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:41.976278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:41.976553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:41.977819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:41.978120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:41.979475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:41.996963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 502/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:41.998621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:42.101880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0792c-b176-46c0-b959-77f87f688703 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:42.139306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:42.140274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:42.140658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:42.141033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:42.148865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:42.150932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:42.165693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:42.166850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:42.168180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:42.169824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 536/1038] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:24:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:42.175271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:42.280993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c85-2c09-492c-9e18-b48cb0672fd6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:42.318581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:42.319510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:42.319814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:42.320094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:42.321473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:42.321785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:42.323507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:42.326410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:42.350244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 503/1039] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:42.351644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:42.367565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:42.463127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c0dea-cbb0-48fa-b55a-3e0a329206f5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:42.497125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:42.497772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:42.498054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:42.498329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:42.567824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:42.569281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:42.583916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:42.672614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 537/1040] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:42.678330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:42.785180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6cd3-478e-4938-bd1d-d3df927d1342 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:42.819520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:42.820360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:42.820642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:42.820974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:42.832150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:42.833594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:42.851263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:42.852213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:42.860663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:42.860663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:42.860663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:42.860663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:42.860663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 504/1041] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:42.861036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:42.961159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cafb07-ac3f-4f4c-836c-564beaee49cb None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:42.974496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:42.997538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:42.998205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:42.998487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:42.998757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:42.999473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:42.999798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.001164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.017090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 538/1042] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:43.018794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.067725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.131091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb9073-3d3c-4d09-9668-dd651acf36be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.168126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.168926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.169208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.169479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.176184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.177622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-707450eb-f94b-4936-94bf-53fd2e0802d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:43.191345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:43.192295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:43.193213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:43.196575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 505/1043] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:43.198996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.232240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750841a-44ca-46f1-a3a5-48248409ed20 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.232567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750841a-44ca-46f1-a3a5-48248409ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.232820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750841a-44ca-46f1-a3a5-48248409ed20 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.234195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 506/1044] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:43.239329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.239614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.239927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.260565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:43.261804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:43.308025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccae13-5aae-428a-9a87-2191554a49d6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.332789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 507/1045] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:43.343578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.344309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.344598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.344872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.345669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:43.345981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.347404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdaa8c-6378-46a5-ba8b-ff5b43e698ce None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.356311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.365241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 539/1046] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:43.365241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e462fb-daee-45d3-a296-5816c0459977 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.365241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e462fb-daee-45d3-a296-5816c0459977 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.365241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e462fb-daee-45d3-a296-5816c0459977 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.365241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 540/1047] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:43.367615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e462fb-daee-45d3-a296-5816c0459977 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.391747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.400128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.464711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de29779-cf83-4e61-ae29-183a75807564 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.476139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e462fb-daee-45d3-a296-5816c0459977 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.497931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.498718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.499000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.499266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.499936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.501284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.509693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.510333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.510612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.510881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.511699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] RBAC: Adding query filter params (role.id=33453989f041479e8edd4baf6eed27c3, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:43.512038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ad216636-b30f-4502-a0cf-6cca2bd06de8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:43.513188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:43.514367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:24:43.515262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:43.516148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.516227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 508/1048] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:43.516227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.516227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.516227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.521000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 541/1049] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=33453989f041479e8edd4baf6eed27c3 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:43.521675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.543322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:43.544588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:43.626279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 509/1050] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:43.635753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.643070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa319e2-846e-4e83-bcc7-d42cf2046b6a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.650231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.681185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.681415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.681884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.682280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.682596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.683375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] RBAC: Adding query filter params (name=image) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:43.683689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.685086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.690351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 542/1051] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:43.695793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.759843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000cd30-daab-46e5-9161-75d4006af2e2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.796136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.797025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.797230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.797480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.809349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:43.812676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959fbc3-6961-4abb-a078-269f636a890d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:43.817474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:43.818399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:43.819261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:43.823907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 510/1052] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:43.824930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.848044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.848710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.849014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.849320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.850035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] RBAC: Adding query filter params (type=image) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:43.850392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.851779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.861789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 543/1053] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:43.862747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:43.890111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.919380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.935235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7ff6-20d6-418a-a172-ca2085a6d78b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.970944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa844a-51b4-43fe-ba0d-767168dbfecd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:43.977810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:43.978492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.979183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.979464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.980250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:43.980921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:43.982626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:43.996940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 511/1054] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:43.997959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:44.007096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:44.007755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:44.008061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:44.008887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:44.009638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:44.010994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:44.034694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 544/1055] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:24:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:24:44.120021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1620e-cbb5-48e1-b200-53b59e3c12f4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:44.164413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:44.166268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:44.166786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:44.167315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] PATH_INFO: `/v3/users/234bac7fc8744eb081a24b78dee6e0d2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:44.182626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] RBAC: Authorizing `identity:get_user(user_id=234bac7fc8744eb081a24b78dee6e0d2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:44.184582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:44.191100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 512/1056] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:43 2022] GET /identity/v3/users/234bac7fc8744eb081a24b78dee6e0d2 => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:44.192648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:44.278651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:44.299631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab9ed3-136d-42be-938b-48d34c557d78 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:44.333937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:44.334591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:44.334874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:44.335152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:44.410234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:44.411689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:44.502006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:44.524951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 545/1057] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:44.530530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:44.537083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:44.649699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6709-c34d-43ef-8249-4b5cc72dab6b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:44.684469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:44.685256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:44.685539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:44.685813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:44.687008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:44.688798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2869a51-522c-47a2-849e-d6bfd3379bc9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:44.695301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 513/1058] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:24:44 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:44.697189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:44.827570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155762-fb69-4ed7-8490-da8d79d2716b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:44.863579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:44.864293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:44.864583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:44.864867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:44.865752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15, user.id=234bac7fc8744eb081a24b78dee6e0d2) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:44.866114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:44.867540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ceeb6b-b266-420d-81e7-93ee75539bc1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:44.879796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 546/1059] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:44 2022] GET /identity/v3/role_assignments?user.id=234bac7fc8744eb081a24b78dee6e0d2&scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:45.604543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e32fc1-8fbf-403f-8676-a60ffba8424e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:45.605128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e32fc1-8fbf-403f-8676-a60ffba8424e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:45.605128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e32fc1-8fbf-403f-8676-a60ffba8424e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:45.606436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 514/1060] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:45.611726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:45.612147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:45.612315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:45.630598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:45.631943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:45.674048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:45.703439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 547/1061] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:45.726318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e32fc1-8fbf-403f-8676-a60ffba8424e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:45.801133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:45.835085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e32fc1-8fbf-403f-8676-a60ffba8424e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:45.870589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:45.871403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:45.871681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:45.871980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:45.882348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:45.891064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:45.892024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:45.892986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:45.895021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 515/1062] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:45.898900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.009945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8b6b-6264-4b39-8022-a1f5563c4509 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.048457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.049231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.049517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.049791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.050560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:46.050877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.052321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245d2b5-b049-4c93-a822-4a699e8102de None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.066938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 548/1063] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:46.068219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.163381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e4fce-172e-4260-b638-3a15f03c9fff None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.163698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e4fce-172e-4260-b638-3a15f03c9fff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.163968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e4fce-172e-4260-b638-3a15f03c9fff None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.165992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 549/1064] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:46.170542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.170829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.171109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.174910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c58560-97e1-4144-8d1f-daf81ff94da5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.188538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:46.189771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:46.211486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.217832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.218740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.219019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.219298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.221379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.231179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.232637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:46.251073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:46.252091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:46.253045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:46.253045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:46.253045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.253045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:46.253045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 516/1065] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:46.258437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.266411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 550/1066] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:46.288281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.371615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcb15-218e-482c-a012-1ab0b95f9389 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.394285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e304c5-6033-47db-9504-2e8f802c8b0b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.413791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.414468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.414794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.415073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.415825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:46.416141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.417498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.438766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 517/1067] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:46.438947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.438947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.438947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.438947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.439957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.472059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.490935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.508378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.510348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.548687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b6c68-a7d2-440d-80aa-da95c017fc97 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.586704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.587499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.587802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.588081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.595696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.597154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:46.614449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:46.620895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:46.621844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 518/1069] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:46.622466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 551/1069] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:24:46.624743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.625463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.697218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.731208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79182f88-4701-45a6-85e7-0428b96c4bd1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.734497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.734766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835458a-cad5-4159-98ae-c6820da6689e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.770727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.771216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.771527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.771848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.771962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.772119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.772213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.772440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.773274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:46.773578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.775749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.782812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.784291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.798940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 552/1070] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:46.799624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:46.802750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:46.803614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:46.804531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:46.804531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:46.804531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.804531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:46.804531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 519/1071] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:46.810299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.906126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e7f0-a9f8-4fdc-928e-114fd2be7a1d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.915405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd070a8a-6b07-4205-ac76-1109fabb2347 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.934077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.948269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.949071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.949353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.949632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.953871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.954779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.955254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.955739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.956696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:46.957076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.958447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217379ab-8fe2-4506-b6da-fcfc2314f319 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.960414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:46.962412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:46.979793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 520/1072] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:46.980749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:46.981705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:46.982595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:46.982595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:46.982595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:46.982595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:46.982595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 553/1073] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:46.987118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509131-0e94-434a-86bf-1449227b916e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.987412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509131-0e94-434a-86bf-1449227b916e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.987678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509131-0e94-434a-86bf-1449227b916e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.988052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.989174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 521/1074] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:46.995281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.995618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.995879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.014704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:47.015941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:47.028616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.091403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 522/1075] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:47.101090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86f9d5-1ce4-482a-b199-4549ac56cc00 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.113711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.142163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.142842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.143120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.143393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.144189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:47.144586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.145872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.160486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 554/1076] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:47.166560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.206289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.231204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3111a48-be95-4537-a9a1-2f43fae937f5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.270515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.271348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.271629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.271947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.278424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd225ea-8bcc-4ba2-8f2b-67cd34af7ba8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.282571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:47.290232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:47.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:47.292372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 523/1077] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:47.297270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.307223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.314557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.315355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.315636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.315943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.323354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.324821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:47.338715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:47.339613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:47.340950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 555/1078] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:47.345639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.401239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabce751-b31e-4709-9f4f-e36bfe255bb1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.438789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.439467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.439772 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.440065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.440644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.440809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:47.441301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.442648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.447517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eac17-bf67-4b92-bd6b-71a23b20a06d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.451583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 524/1079] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:47.458104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.486756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.487423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.487702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.488004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.488843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:47.489148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.490967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.504391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 556/1080] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:47.510235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.525167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.559084 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c580b-eb4e-4ea1-b269-66e19e259c88 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.591456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.592360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.592650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.592924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] PATH_INFO: `/v3/users/234bac7fc8744eb081a24b78dee6e0d2` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.594129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] RBAC: Authorizing `identity:get_user(user_id=234bac7fc8744eb081a24b78dee6e0d2)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.595556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.597276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 525/1081] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/users/234bac7fc8744eb081a24b78dee6e0d2 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:47.603361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.607245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07babfc0-19ba-4ada-aa82-0b1beddf866c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.640415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.641303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.641643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.641924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/33453989f041479e8edd4baf6eed27c3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.644220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=e441b994217b4b81be762246ef726545, role_id=33453989f041479e8edd4baf6eed27c3)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.645646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.658513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 557/1082] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:47 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/e441b994217b4b81be762246ef726545/roles/33453989f041479e8edd4baf6eed27c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:24:47.703167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3bb4-8487-4a49-a7ca-d49fb25abb04 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.739144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.740051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.740331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.740600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.741793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.743880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2af984-102b-4abc-8e1f-c5bed5c1d336 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.745611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 526/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:24:47 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:47.751827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:47.771120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:47.858591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bed666-d0f2-4cbc-8c26-c1eb3c8c6d2e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:47.890991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:47.891913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:47.892192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:47.892461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15/users/234bac7fc8744eb081a24b78dee6e0d2/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:47.894392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] RBAC: Authorizing `identity:create_grant(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15, user_id=234bac7fc8744eb081a24b78dee6e0d2, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.895834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164a608-5f66-4706-b5ca-116adf5bad5e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.909023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 558/1084] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:47 2022] PUT /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15/users/234bac7fc8744eb081a24b78dee6e0d2/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:24:49.689764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f772-0fc4-4243-93d4-24d28f59ea1d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.690285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f772-0fc4-4243-93d4-24d28f59ea1d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.690285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f772-0fc4-4243-93d4-24d28f59ea1d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.691857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 527/1085] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:49.697243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae2f0-3e27-460f-b0ea-b2def213faad None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.697638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae2f0-3e27-460f-b0ea-b2def213faad None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.697788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae2f0-3e27-460f-b0ea-b2def213faad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.714110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-97fae2f0-3e27-460f-b0ea-b2def213faad None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:49.715282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-97fae2f0-3e27-460f-b0ea-b2def213faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:49.780275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 559/1086] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:49.803572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19f772-0fc4-4243-93d4-24d28f59ea1d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:49.807957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:49.904235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19f772-0fc4-4243-93d4-24d28f59ea1d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:49.921161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e5e7-90d6-4a55-8308-4dd61d2fec26 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.921448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e5e7-90d6-4a55-8308-4dd61d2fec26 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.921722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61e5e7-90d6-4a55-8308-4dd61d2fec26 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.923102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 560/1087] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:49.927939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274393db-f48c-4452-b133-34349cd5e387 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.928226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274393db-f48c-4452-b133-34349cd5e387 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.928508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274393db-f48c-4452-b133-34349cd5e387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.938068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:49.938881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.939163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.939436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.939880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:49.946222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-274393db-f48c-4452-b133-34349cd5e387 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:49.947564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-274393db-f48c-4452-b133-34349cd5e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:49.951009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:49.959109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:49.960238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:49.961125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 528/1088] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jul 20 15:24:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:24:49.966653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.014179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 561/1089] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:24:50.028006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.037499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274393db-f48c-4452-b133-34349cd5e387 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.065964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f11ab-53cc-4d60-be9f-4b187a1448c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.099009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.099668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.099983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.100256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.100993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:50.101295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.103170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.110831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 529/1090] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:24:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:50.116469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.135317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274393db-f48c-4452-b133-34349cd5e387 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.170857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.171641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.171954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.172231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.174806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.183039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:24:50.190685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:50.191696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:24:50.192609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 562/1091] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:50.197712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.221587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7b48-5478-4dce-ace0-16a38c063a72 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.246989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.256230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.257002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.257281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.257552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.268144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.269585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:50.286840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:50.287794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:50.288705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:50.288705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:50.288705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:50.288705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:50.288705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 530/1092] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:50.294169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.296854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e159f-dc0c-4c62-9944-5296f8f8e882 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.330024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.330693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.330978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.331253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.332027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:50.332340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.333707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.342253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 563/1093] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:50.347920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.391660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082d880-0dc2-424c-9b88-741a94b0416a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.406120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.429308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.430252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.430535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.430809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.431689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:50.432026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.433479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.445355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 531/1094] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:50.450848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcc249-c8a3-4733-b31c-45fac28b65dd None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.451283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.483691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.484588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.484865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.485133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] PATH_INFO: `/v3/users/234bac7fc8744eb081a24b78dee6e0d2` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.486320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] RBAC: Authorizing `identity:get_user(user_id=234bac7fc8744eb081a24b78dee6e0d2)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.487732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.489199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.489534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 564/1095] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/users/234bac7fc8744eb081a24b78dee6e0d2 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:50.495424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.549440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9431d-cc15-412b-a311-65ce12a8cfcc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.582889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.583673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.583985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.584260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.591278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.592746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.593509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3ac2-bf31-4c3c-bd75-9ffe66d03d0a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:24:50.607355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:50.608425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 532/1096] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:50.614187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.619808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.628877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.629531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.629813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.630111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.699685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.701450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.709392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.713501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c802af4-a4af-4d88-8484-1097d557091d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.749076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.749741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.750022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.750298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.751130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:50.751439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.752890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.769507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 533/1097] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:50.775283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.798870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 565/1098] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:24:50.804724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.831858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.872964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f48578-a2c5-427f-a952-b32c3d306e58 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.904938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcd377-5ec3-49fe-b7f8-4b327edab009 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.906543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.907186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.907463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.907734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.933049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.938593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.939438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.939714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.940014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.941176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.942592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.944294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 566/1099] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:50.950427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.977440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.978927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.040775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.047679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d163af4-1916-4c83-a32e-4235cd41ad1e None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.077070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 534/1100] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:24:51.082660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.085686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.086383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.086673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.086951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.087811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15, user.id=234bac7fc8744eb081a24b78dee6e0d2) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:51.088130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.089554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.095802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 567/1101] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/role_assignments?user.id=234bac7fc8744eb081a24b78dee6e0d2&scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:24:51.139889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.185401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392ca4-2f03-4167-8309-fb3d7bb76d76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.220448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.221153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.221433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.221707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.233156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.234690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:24:51.253190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:51.254273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:51.255207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:24:51.255207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:24:51.255207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:24:51.255207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:51.255207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 535/1102] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:51.260994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.361332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aac15-6031-4c9c-8f5f-5f46bffe7cee None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.394449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.395106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.395385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.395660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.396409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:51.396716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.398068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.409509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 568/1103] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:51.415645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.516927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91198-1bec-41cc-a54b-fe19180bf497 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.525902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.551021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.551843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.552130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.552409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.559573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.561087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:24:51.574773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:24:51.575636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:24:51.576541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 536/1104] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:24:51.581843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.601078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.682944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297f7e-3a29-4f0a-9d9b-ce0047a1e3f6 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.718754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.719412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.719695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.720002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.720782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:51.721093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.722516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a8687-9361-4f9d-8177-740bca38b9c4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.734302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 569/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:51.740262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.839254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a426f62-d9ea-4c6f-8c3a-d44dc04314ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.872472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.873123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.873413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.873692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.874521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] RBAC: Adding query filter params (role.id=33453989f041479e8edd4baf6eed27c3, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=e441b994217b4b81be762246ef726545) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:51.874836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.876291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474ecd9-b0a2-42c3-846e-6821a014d0bc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:51.882096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 537/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/role_assignments?user.id=e441b994217b4b81be762246ef726545&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=33453989f041479e8edd4baf6eed27c3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:53.149662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2a936-5796-41ad-ad89-7e16130bf8ff None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:53.150191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2a936-5796-41ad-ad89-7e16130bf8ff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:53.150281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2a936-5796-41ad-ad89-7e16130bf8ff None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:53.151602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 570/1107] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:53.156730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:53.157167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:53.157286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:53.173677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:53.174854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:53.200727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:53.240856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 538/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:53.266011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2a936-5796-41ad-ad89-7e16130bf8ff None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:53.342950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:53.364716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2a936-5796-41ad-ad89-7e16130bf8ff None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:53.397745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:53.398520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:53.398826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:53.399102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:53.400388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:53.401777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9f5a-1f38-4864-a158-fcf118af3847 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:53.403393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 571/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:24:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:53.409095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:53.507049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd621cf-13a9-4fdd-a10d-a78d035c52e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:53.539261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:53.540463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:53.540748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:53.541019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:53.541716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:53.543131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22aa55-d197-4193-8c7b-a9e7958c9741 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:53.562693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 539/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:24:53 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:24:55.603846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fc71-d197-40f2-aee8-31d124f82834 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.604563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fc71-d197-40f2-aee8-31d124f82834 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.604563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fc71-d197-40f2-aee8-31d124f82834 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.605770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 572/1111] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:55.610890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.611285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.611439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.628522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:55.629692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:55.643109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:55.696333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 540/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:55.719291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fc71-d197-40f2-aee8-31d124f82834 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:55.817505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fc71-d197-40f2-aee8-31d124f82834 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:55.849189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:55.850023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.850302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.850577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] PATH_INFO: `/v3/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.852547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] RBAC: Authorizing `identity:get_role(role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:55.853898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:55.855228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 573/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:55 2022] GET /identity/v3/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:55.860926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:55.956501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d66e0-398e-42c5-9408-b0bf25485a80 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:55.988921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:55.989770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.990047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.990318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] PATH_INFO: `/v3/users/b320b3c16ca94a318115bc15f7addb36` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.003480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] RBAC: Authorizing `identity:get_user(user_id=b320b3c16ca94a318115bc15f7addb36)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:56.004925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:56.006516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 541/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:55 2022] GET /identity/v3/users/b320b3c16ca94a318115bc15f7addb36 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:56.012089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:56.027136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.108085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95929-52df-4c70-b6b3-a04ba1e0a23f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:56.140386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:56.141015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.141302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.141577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.205027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:56.206441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:56.220726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.302009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 574/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:24:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:24:56.307817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:56.316396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.405171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da5d8f-dec7-470d-99b0-42732af4ded4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:56.438356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:56.439211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.439490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.439784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.440941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:56.442344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9694c-2492-4a9d-9970-c1febdb684fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:56.444042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 542/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:24:56 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:56.451008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:56.546702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20c1f-7e2c-4349-bb0e-df04b863cc3d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:56.560673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.581021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:56.581662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.581943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.582215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.583034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15, user.id=b320b3c16ca94a318115bc15f7addb36) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:24:56.583343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:56.584778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653abc71-0543-4024-afb3-61fe9ae13958 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:56.590629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 575/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:24:56 2022] GET /identity/v3/role_assignments?user.id=b320b3c16ca94a318115bc15f7addb36&scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:58.566589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ab41-2a5d-49ed-996f-9245da86d879 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.567145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ab41-2a5d-49ed-996f-9245da86d879 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.567145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108ab41-2a5d-49ed-996f-9245da86d879 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.568548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 543/1118] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:58.574482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.574903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.575049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.591398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:58.592609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:58.656541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 576/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:24:58.679589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108ab41-2a5d-49ed-996f-9245da86d879 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:58.754751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:58.779673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108ab41-2a5d-49ed-996f-9245da86d879 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:58.813116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:58.813966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.814248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.814524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] PATH_INFO: `/v3/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.816007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] RBAC: Authorizing `identity:get_role(role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:58.817367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:58.818725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 544/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:58 2022] GET /identity/v3/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:58.824841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:58.924428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6dcaa-faf8-4109-a2a7-12978912f199 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:58.933911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:58.958010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:58.958875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.959159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.959438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] PATH_INFO: `/v3/users/b320b3c16ca94a318115bc15f7addb36` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.960648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] RBAC: Authorizing `identity:get_user(user_id=b320b3c16ca94a318115bc15f7addb36)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:58.962075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:58.963728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 577/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:24:58 2022] GET /identity/v3/users/b320b3c16ca94a318115bc15f7addb36 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:58.969508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:59.068157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbb4c-495b-4039-9069-f20c145c710f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:59.100819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:59.101664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:59.101944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:59.102217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:59.103383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:59.105356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d2242-fd1d-4a9d-86c5-9111b65574b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:59.107015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 545/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:24:58 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:24:59.113095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:59.212414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909299d-68a5-4b30-a33d-b5083808234a None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:59.246174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:59.247053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:59.247337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:59.247612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15/users/b320b3c16ca94a318115bc15f7addb36/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:59.249840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15, user_id=b320b3c16ca94a318115bc15f7addb36, role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:59.251273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbfc7b-da59-4c3c-a396-b93feb751dc4 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:59.263662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 578/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:24:59 2022] PUT /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15/users/b320b3c16ca94a318115bc15f7addb36/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:01.307628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156a93-dc30-4dc5-987e-2fbb0826d120 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:01.308202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156a93-dc30-4dc5-987e-2fbb0826d120 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:01.308202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156a93-dc30-4dc5-987e-2fbb0826d120 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:01.309829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 546/1124] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:01.315272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:01.315716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:01.315855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:01.332919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:01.334105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:01.361927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:01.400570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 579/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:01.425318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156a93-dc30-4dc5-987e-2fbb0826d120 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:01.437954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:01.526800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156a93-dc30-4dc5-987e-2fbb0826d120 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:01.560628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:01.561415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:01.561698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:01.561972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] PATH_INFO: `/v3/roles/0407d79066964d7b95f4bd3fc21f4067` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:01.563434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] RBAC: Authorizing `identity:get_role(role_id=0407d79066964d7b95f4bd3fc21f4067)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:01.565709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:01.567092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 547/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:01 2022] GET /identity/v3/roles/0407d79066964d7b95f4bd3fc21f4067 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:01.573821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:01.670009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a84f-54ab-4bf2-bf87-2871f2f47bce None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:01.702760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:01.703614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:01.703922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:01.704197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] PATH_INFO: `/v3/users/b320b3c16ca94a318115bc15f7addb36` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:01.705374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] RBAC: Authorizing `identity:get_user(user_id=b320b3c16ca94a318115bc15f7addb36)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:01.706813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:01.708454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 580/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:01 2022] GET /identity/v3/users/b320b3c16ca94a318115bc15f7addb36 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:01.714151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:01.794918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:01.816706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ccc3f-8b30-49e8-aebc-fbbf18b8d235 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:01.850517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:01.851513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:01.851936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:01.852342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:01.941709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:01.943875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:02.045015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:02.075486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 548/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:02.081380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:02.116041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:02.182010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93985179-3b44-4263-823a-df8f363305a5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:02.215710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550100-d065-4221-ba13-f1be09821025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:02.216587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550100-d065-4221-ba13-f1be09821025 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.216868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550100-d065-4221-ba13-f1be09821025 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.217143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550100-d065-4221-ba13-f1be09821025 None admin] PATH_INFO: `/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.218327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550100-d065-4221-ba13-f1be09821025 None admin] RBAC: Authorizing `identity:get_project(project_id=ae87c0a1efcc4253b1fb6bb260c6ef15)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:02.219782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550100-d065-4221-ba13-f1be09821025 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:02.221456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 581/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:02 2022] GET /identity/v3/projects/ae87c0a1efcc4253b1fb6bb260c6ef15 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:02.227639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:02.325654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632c2d7-ef65-47ed-8595-2bcc07cd6532 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:02.358272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:02.358916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.359201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.359478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.360334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] RBAC: Adding query filter params (role.id=0407d79066964d7b95f4bd3fc21f4067, scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15, user.id=b320b3c16ca94a318115bc15f7addb36) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:02.360650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:02.362071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211d616-a532-417e-b6d6-3b82c3cc73ea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:02.367838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 549/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:25:02 2022] GET /identity/v3/role_assignments?user.id=b320b3c16ca94a318115bc15f7addb36&scope.project.id=ae87c0a1efcc4253b1fb6bb260c6ef15&role.id=0407d79066964d7b95f4bd3fc21f4067 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:04.442347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460713-3f18-4f13-b7b9-0ab162bee374 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:04.442858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460713-3f18-4f13-b7b9-0ab162bee374 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:04.442952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460713-3f18-4f13-b7b9-0ab162bee374 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:04.444328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 582/1131] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:04.450145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:04.450602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:04.450705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:04.467052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:04.468274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:04.505975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:04.547319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 550/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:25:04.578205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb460713-3f18-4f13-b7b9-0ab162bee374 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:04.683791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb460713-3f18-4f13-b7b9-0ab162bee374 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:04.685212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:04.718101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:04.718886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:04.719168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:04.719441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:04.720640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:04.722077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57250ed5-0a02-49c0-8d45-cbe09527b25d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:04.723686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 583/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:25:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:04.730114 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:04.834988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cfa74-a2a9-4aa7-93de-fa322b86e766 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:04.869812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:04.870476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:04.870753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:04.871022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:04.871730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:04.873244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2512c9c-9c0d-4294-964c-344333192082 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:04.897180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 551/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:25:04 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:25:06.963352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f437-ec55-4c57-94b1-3652287a8649 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:06.963984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f437-ec55-4c57-94b1-3652287a8649 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:06.963984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430f437-ec55-4c57-94b1-3652287a8649 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:06.965308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 584/1135] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:06.970470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:06.970907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:06.971013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:06.987437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:06.988643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:07.005682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:07.055031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 552/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:07.079930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430f437-ec55-4c57-94b1-3652287a8649 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:07.178747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430f437-ec55-4c57-94b1-3652287a8649 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:07.211923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:07.212761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:07.213596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:07.213819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:07.214996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:07.216457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad74a56-b273-4443-b833-a11083d91796 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:07.218089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 585/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:25:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:07.223838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:07.321342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7aa27-4c21-446b-ad7b-691819c02831 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:07.354305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:07.354976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:07.355260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:07.355536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:07.356264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:07.357693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f073da-707d-417c-8e98-ce5af8f569c3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:07.378076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 553/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:25:07 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:09.444084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa57b-7daf-4a3e-b25a-50a8e0d1075e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:09.444499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa57b-7daf-4a3e-b25a-50a8e0d1075e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:09.444634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa57b-7daf-4a3e-b25a-50a8e0d1075e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:09.446032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 586/1139] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:09.451177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:09.451585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:09.451728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:09.468109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:09.469283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:09.472102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:09.535297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 554/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:09.558624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa57b-7daf-4a3e-b25a-50a8e0d1075e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:09.598418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:09.658406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa57b-7daf-4a3e-b25a-50a8e0d1075e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:09.691148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:09.691897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:09.692180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:09.692453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:09.702249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:09.709816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:09.710979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:09.712279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 587/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:09.717486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:09.812634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414fa4c-0a91-4d9f-b228-a14971e3821f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:09.845148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:09.845807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:09.846091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:09.846368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:09.847107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:09.847414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:09.848818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:09.857241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 555/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:09.862758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:09.956375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:09.960527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a567b-500d-4094-82a1-3322402ad810 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:09.993348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:09.994243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:09.994521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:09.994796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] PATH_INFO: `/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:10.007410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] RBAC: Authorizing `identity:get_user(user_id=0d2dad2f4ad04a6fb1a660b3e8896a1d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:10.008859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:10.010992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 588/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:09 2022] GET /identity/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:10.016676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:10.019550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:10.114393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6567ec-d28c-4450-a286-dd54aebd21f0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:10.148313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:10.148954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:10.149237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:10.149512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:10.214600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:10.216023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:10.217844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:10.313705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 556/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:10.319864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:10.422497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f840-1bc7-40be-853a-0f00283d6029 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:10.455615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:10.456489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:10.456772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:10.457044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] PATH_INFO: `/v3/projects/268448685f2f49298b7f08fc1d8158fb` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:10.458214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] RBAC: Authorizing `identity:get_project(project_id=268448685f2f49298b7f08fc1d8158fb)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:10.459645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d79be-2c47-44bf-b086-8c7ee5e67f48 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:10.461356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 589/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:10 2022] GET /identity/v3/projects/268448685f2f49298b7f08fc1d8158fb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:10.468118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:10.563690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8e97f-c35e-49e6-b1ce-b25ed7e23394 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:10.565197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:10.599157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:10.599888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:10.600167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:10.600434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:10.601675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=268448685f2f49298b7f08fc1d8158fb, user.id=0d2dad2f4ad04a6fb1a660b3e8896a1d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:10.601988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:10.604036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a956c8-b815-43d8-9e70-3d349da0e5e2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:10.610558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 557/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:25:10 2022] GET /identity/v3/role_assignments?user.id=0d2dad2f4ad04a6fb1a660b3e8896a1d&scope.project.id=268448685f2f49298b7f08fc1d8158fb&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:12.600678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be005-fb7e-4e1d-a055-42f74a9b1355 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:12.601231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be005-fb7e-4e1d-a055-42f74a9b1355 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:12.601231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0be005-fb7e-4e1d-a055-42f74a9b1355 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:12.602591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 590/1147] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:12.608354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:12.608746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:12.608914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:12.625522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:12.626724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:12.690867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 558/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:12.713792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0be005-fb7e-4e1d-a055-42f74a9b1355 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:12.739073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:12.817064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0be005-fb7e-4e1d-a055-42f74a9b1355 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:12.850021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:12.850797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:12.851081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:12.851354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:12.861794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:12.869604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:12.870688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:12.871753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 591/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:12.876826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:12.967146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:12.973371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6584-3fad-4ffb-9d65-9acecd3be0e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:13.006510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:13.007174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.007455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.007728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.008535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:13.008810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:13.010694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:13.018832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 559/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:13.024705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:13.105298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:13.120771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb3438-abde-40e7-9b90-d084df838c7b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:13.153316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:13.154154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.154431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.154697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] PATH_INFO: `/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.155870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] RBAC: Authorizing `identity:get_user(user_id=0d2dad2f4ad04a6fb1a660b3e8896a1d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:13.157287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:13.158897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 592/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:13 2022] GET /identity/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:13.165145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:13.261449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b2d19-1bb9-4080-a632-3cf7e1e1a01f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:13.293619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:13.294460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.294742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.295015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] PATH_INFO: `/v3/projects/268448685f2f49298b7f08fc1d8158fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.296204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] RBAC: Authorizing `identity:get_project(project_id=268448685f2f49298b7f08fc1d8158fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:13.297617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd0c3b-d622-4114-b7c3-83ac6f500f15 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:13.299277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 560/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:13 2022] GET /identity/v3/projects/268448685f2f49298b7f08fc1d8158fb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:13.305270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:13.411720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514a176-4f40-485b-b9d0-e6494194ef5b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:13.445302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:13.446185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.446466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.446739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] PATH_INFO: `/v3/projects/268448685f2f49298b7f08fc1d8158fb/users/0d2dad2f4ad04a6fb1a660b3e8896a1d/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.449273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] RBAC: Authorizing `identity:create_grant(project_id=268448685f2f49298b7f08fc1d8158fb, user_id=0d2dad2f4ad04a6fb1a660b3e8896a1d, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:13.450697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bde06-16de-4506-a714-03f685a1354b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:13.463878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 593/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:25:13 2022] PUT /identity/v3/projects/268448685f2f49298b7f08fc1d8158fb/users/0d2dad2f4ad04a6fb1a660b3e8896a1d/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:15.482699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb30a0-701d-4dbc-a5c4-1f92f5859c2e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:15.483208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb30a0-701d-4dbc-a5c4-1f92f5859c2e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:15.483298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb30a0-701d-4dbc-a5c4-1f92f5859c2e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:15.484670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 561/1154] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:15.489771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:15.490183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:15.490337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:15.506960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:15.508174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:15.515845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:15.574775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 594/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:15.598950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb30a0-701d-4dbc-a5c4-1f92f5859c2e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:15.632089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:15.705626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb30a0-701d-4dbc-a5c4-1f92f5859c2e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:15.737390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:15.738170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:15.738456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:15.738730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:15.749973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:15.757583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:15.758666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:25:15.760357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 562/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:15.765669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:15.861041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bac9aa-d0bd-4bbd-99ab-7443c3c819f1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:15.894353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:15.895011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:15.895293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:15.895565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:15.896338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:15.896645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:15.898003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:15.905906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 595/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:15.911560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:15.994484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.009081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3f416-38e1-421a-830c-d0796a198677 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:16.041960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:16.042811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.043095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.043374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] PATH_INFO: `/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.044579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] RBAC: Authorizing `identity:get_user(user_id=0d2dad2f4ad04a6fb1a660b3e8896a1d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:16.046006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:16.047598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 563/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:15 2022] GET /identity/v3/users/0d2dad2f4ad04a6fb1a660b3e8896a1d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:16.053289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:16.057640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.151479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d26b-b395-429e-8cf9-985637cb502b None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:16.184357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:16.184999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.185286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.185564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.250475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:16.251890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:16.254402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.346342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 596/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:16.352225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:16.449949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb3ee-4538-4a99-9552-e18769f30636 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:16.482538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:16.483387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.483958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.483958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] PATH_INFO: `/v3/projects/268448685f2f49298b7f08fc1d8158fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.485128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] RBAC: Authorizing `identity:get_project(project_id=268448685f2f49298b7f08fc1d8158fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:16.486542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8535e23-aac1-41cd-b8f2-9c585b8c99ed None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:16.488260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 564/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:16 2022] GET /identity/v3/projects/268448685f2f49298b7f08fc1d8158fb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:16.495055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:16.590024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0ea0b-5a79-422e-b0c2-e00c179bfbf2 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:16.592281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.623256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:16.623944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.624228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.624497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.625320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=268448685f2f49298b7f08fc1d8158fb, user.id=0d2dad2f4ad04a6fb1a660b3e8896a1d) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:16.625628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:16.627064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d292a-694e-464e-a26f-da845d6180aa None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:16.633858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 597/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:25:16 2022] GET /identity/v3/role_assignments?user.id=0d2dad2f4ad04a6fb1a660b3e8896a1d&scope.project.id=268448685f2f49298b7f08fc1d8158fb&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:18.675647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556683b-2ba0-4b47-929c-709544ebf0df None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:18.676203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556683b-2ba0-4b47-929c-709544ebf0df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:18.676203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556683b-2ba0-4b47-929c-709544ebf0df None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:18.677513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 565/1162] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:18.683296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:18.683675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:18.683902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:18.700696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:18.701862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:18.772188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 598/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:18.794434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556683b-2ba0-4b47-929c-709544ebf0df None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:18.818705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:18.897370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556683b-2ba0-4b47-929c-709544ebf0df None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:18.931422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:18.932030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:18.932311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:18.932581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:18.997847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:18.999281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:19.021606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:19.095597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 566/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:19.101578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:19.193722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:19.198118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb8060-aa7e-41ab-a93b-20b9f430bd73 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:19.231016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:19.231825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:19.232109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:19.232407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:19.238868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:19.240350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-9c5ab080-498b-47a3-98cb-e019db1ce514 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:19.254347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:19.255553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:19.256793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 599/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:25:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:19.262031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:19.357597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113174f2-4da4-4b62-be04-6eb93c2f108a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:19.379993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:19.392135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:19.392791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:19.393074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:19.393352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:19.394067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:19.394374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:19.395734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cde597-0a44-4378-afb5-429aacb9bd7a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:19.402755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 567/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jul 20 15:25:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:21.380664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ec3e-9480-4a3e-b643-2a7a13814f8f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:21.381208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ec3e-9480-4a3e-b643-2a7a13814f8f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:21.381208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ec3e-9480-4a3e-b643-2a7a13814f8f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:21.382596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 600/1167] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:21.387625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc3c9-54aa-4966-bc44-fedff2dfd380 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:21.388054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc3c9-54aa-4966-bc44-fedff2dfd380 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:21.388206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbc3c9-54aa-4966-bc44-fedff2dfd380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:21.404482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d4fbc3c9-54aa-4966-bc44-fedff2dfd380 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:21.405649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d4fbc3c9-54aa-4966-bc44-fedff2dfd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:21.470292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 568/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:21.492687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ec3e-9480-4a3e-b643-2a7a13814f8f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:21.592338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ec3e-9480-4a3e-b643-2a7a13814f8f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:21.625374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:21.626016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:21.626302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:21.626581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:21.627329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:21.628713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58922a69-f4c0-4863-a5a9-58ea68813e5f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:21.642806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 601/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Wed Jul 20 15:25:21 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:23.662584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6201d1-ff16-475c-ae2f-0bf5d3fb7a1e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:23.663134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6201d1-ff16-475c-ae2f-0bf5d3fb7a1e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:23.663134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6201d1-ff16-475c-ae2f-0bf5d3fb7a1e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:23.664543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 569/1170] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:23.669853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:23.670245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:23.670401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:23.687454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:23.688607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:23.702001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:23.755138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 602/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:23.779328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6201d1-ff16-475c-ae2f-0bf5d3fb7a1e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:23.877144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6201d1-ff16-475c-ae2f-0bf5d3fb7a1e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:23.893480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:23.910994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:23.911869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:23.912154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:23.912429 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] PATH_INFO: `/v3/domains/fb3af3d7530a4ed187ee6142379acea6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:23.922586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fb3af3d7530a4ed187ee6142379acea6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:23.924050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c08a-b7d0-4910-8131-affcf84d3d28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:23.925647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 570/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Jul 20 15:25:23 2022] GET /identity/v3/domains/fb3af3d7530a4ed187ee6142379acea6 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:23.931995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:24.030170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033462c-142c-403a-8363-07b8c51c64b1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:24.062585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:24.063232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:24.063513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:24.063811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:24.064504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:24.065923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f8170-06cc-4612-8bf2-ed2b3bbc9602 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:24.084967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:24.092171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 603/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:25:23 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:25:26.125224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62c7a-ef27-48be-ba76-912333f8d558 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:26.125854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62c7a-ef27-48be-ba76-912333f8d558 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:26.125854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62c7a-ef27-48be-ba76-912333f8d558 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:26.127155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 571/1174] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:26.132416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:26.132835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:26.132974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:26.150670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:26.151870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:26.217642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 604/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:26.242756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c62c7a-ef27-48be-ba76-912333f8d558 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:26.342078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c62c7a-ef27-48be-ba76-912333f8d558 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:26.374730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:26.375635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:26.375906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:26.376189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] PATH_INFO: `/v3/domains/fb3af3d7530a4ed187ee6142379acea6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:26.377368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fb3af3d7530a4ed187ee6142379acea6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:26.378795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aecd6-f9dd-411c-bfcf-385419f64068 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:26.380417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 572/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Jul 20 15:25:26 2022] GET /identity/v3/domains/fb3af3d7530a4ed187ee6142379acea6 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:26.386633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:26.482749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4665a0f-2eb7-444f-b677-d65d63caf6dc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:26.510513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:26.516667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:26.517337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:26.517619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:26.517893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:26.518676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:26.520132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4a105-946c-4902-b916-aac460deb39a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:26.539144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 605/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:25:26 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:25:28.634338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d096-9b5d-4daa-bd86-94660cd09ffa None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.634950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d096-9b5d-4daa-bd86-94660cd09ffa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.634950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758d096-9b5d-4daa-bd86-94660cd09ffa None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.636301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 573/1178] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:28.641427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.641816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.641976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.658838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:28.660063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.724375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 606/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:28.748092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758d096-9b5d-4daa-bd86-94660cd09ffa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:28.794581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:28.857005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758d096-9b5d-4daa-bd86-94660cd09ffa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:28.897039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.897830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.898111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.898381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.909250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:28.916840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:28.918019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:25:28.918947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 574/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:28.924216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.019948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c910d-fa0d-4f53-ae2c-9e70ce3211a7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.036100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.054208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.054871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.055154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.055432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.056729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:29.057039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.058408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.066272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 607/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:29.072645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.168297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244836-0a20-4358-809c-e1d608cc2b53 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.170551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.206806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.208406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.208406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.208406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] PATH_INFO: `/v3/users/1b134ac3f6f34892a62dcea74e026c33` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.222979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] RBAC: Authorizing `identity:get_user(user_id=1b134ac3f6f34892a62dcea74e026c33)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.224466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.226139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 575/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/users/1b134ac3f6f34892a62dcea74e026c33 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:29.232259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.332577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03489191-0616-48b6-b692-814d42d3a39c None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.363883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.364511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.364789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.365060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.408737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.432286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.433721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.527180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 608/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:29.533182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.633919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df055ef-5ab7-4ab7-b92b-2260f4b91c4b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.667110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.667989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.668273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.668575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] PATH_INFO: `/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.669770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] RBAC: Authorizing `identity:get_project(project_id=7b93d88c15094f44b25f8d85a7dbeb37)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.671190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaac82-1f47-4340-9e1f-b7e3a4151876 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.672909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 576/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:29.679138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:29.755262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:29.776748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b8948-d4a9-40eb-88af-b6bbab05d085 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:29.809697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:29.810344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:29.810628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:29.810904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:29.811733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=7b93d88c15094f44b25f8d85a7dbeb37, user.id=1b134ac3f6f34892a62dcea74e026c33) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:29.812072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:29.813492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1254aa7-133d-4985-976e-c41322668864 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:29.818873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 609/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:25:29 2022] GET /identity/v3/role_assignments?user.id=1b134ac3f6f34892a62dcea74e026c33&scope.project.id=7b93d88c15094f44b25f8d85a7dbeb37&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:31.809680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315789-5c3c-4a64-af11-21d2157c09c9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:31.810184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315789-5c3c-4a64-af11-21d2157c09c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:31.810184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315789-5c3c-4a64-af11-21d2157c09c9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:31.811511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 577/1186] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:31.816594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:31.816982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:31.817159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:31.834052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:31.835228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:31.901102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 610/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:31.923472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07315789-5c3c-4a64-af11-21d2157c09c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:31.953368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:32.023230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07315789-5c3c-4a64-af11-21d2157c09c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:32.056303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:32.057062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:32.057343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:32.057615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:32.067388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:32.075482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:32.076654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:25:32.077791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 578/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:32.082719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:32.150031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:32.178495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc97b-6700-4829-9e7f-dd5853e5a050 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:32.212353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:32.213018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:32.213303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:32.213583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:32.214330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:32.214639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:32.216033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:32.223995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 611/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:32.229823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:32.318752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:32.329018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7475fa-bceb-42f3-9df1-b8a7aff1e8eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:32.361094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:32.361951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:32.362234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:32.362512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] PATH_INFO: `/v3/users/1b134ac3f6f34892a62dcea74e026c33` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:32.363686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] RBAC: Authorizing `identity:get_user(user_id=1b134ac3f6f34892a62dcea74e026c33)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:32.365141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:32.366758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 579/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:32 2022] GET /identity/v3/users/1b134ac3f6f34892a62dcea74e026c33 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:32.373363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:32.472145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546270-9620-42c9-a59f-738c946f3bf0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:32.504777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:32.505617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:32.505894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:32.506161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] PATH_INFO: `/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:32.507302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] RBAC: Authorizing `identity:get_project(project_id=7b93d88c15094f44b25f8d85a7dbeb37)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:32.508753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6db088-1445-4a7e-8802-5593dd6fff4e None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:32.510426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 612/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:32 2022] GET /identity/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:32.516377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:32.613161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91889-ec74-4dfd-a426-9b26a968378a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:32.645631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:32.646508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:32.646793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:32.647068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] PATH_INFO: `/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37/users/1b134ac3f6f34892a62dcea74e026c33/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:32.649033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b93d88c15094f44b25f8d85a7dbeb37, user_id=1b134ac3f6f34892a62dcea74e026c33, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:32.650478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9629ae-52b6-43a7-924e-834134b053f5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:32.662775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 580/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:25:32 2022] PUT /identity/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37/users/1b134ac3f6f34892a62dcea74e026c33/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:25:34.672024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd125-1a4c-4487-bef4-59b9258c088b None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:34.672566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd125-1a4c-4487-bef4-59b9258c088b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:34.672666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cd125-1a4c-4487-bef4-59b9258c088b None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:34.673952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 613/1193] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:34.679086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081657-3794-4054-81cb-0299acc8c146 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:34.679454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081657-3794-4054-81cb-0299acc8c146 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:34.679601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081657-3794-4054-81cb-0299acc8c146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:34.696633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-29081657-3794-4054-81cb-0299acc8c146 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:34.697814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-29081657-3794-4054-81cb-0299acc8c146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:34.710590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:34.764833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 581/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:34.787805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cd125-1a4c-4487-bef4-59b9258c088b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:34.791195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:34.888850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cd125-1a4c-4487-bef4-59b9258c088b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:34.925404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:34.926184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:34.926468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:34.926748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:34.937277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:25:34.944834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:25:34.945884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:25:34.946837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 614/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:25:34.952124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29081657-3794-4054-81cb-0299acc8c146 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:35.049464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29081657-3794-4054-81cb-0299acc8c146 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:35.081923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:35.082584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:35.082867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:35.083141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:35.083917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:35.084225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:35.085579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:35.093985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 582/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:35.099743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:35.160318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:35.198590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24017af0-7cdd-4162-aef3-7efe9dd8cd48 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:35.231510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:35.232388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:35.232670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:35.232945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] PATH_INFO: `/v3/users/1b134ac3f6f34892a62dcea74e026c33` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:35.234113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] RBAC: Authorizing `identity:get_user(user_id=1b134ac3f6f34892a62dcea74e026c33)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:35.235534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:35.237151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 615/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jul 20 15:25:35 2022] GET /identity/v3/users/1b134ac3f6f34892a62dcea74e026c33 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:35.242778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:35.255621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:35.342581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be5a5-cdcc-4ea5-8dd0-15b441f7ba7e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:35.374894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:35.375529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:35.375835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:35.376109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:35.440730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:35.442153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:35.452860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:35.536494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 583/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:25:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:35.543034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:35.644195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94a5d-a669-413c-905e-51471f3dea76 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:35.671407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:35.677617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:35.678463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:35.678738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:35.679005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] PATH_INFO: `/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:35.680207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] RBAC: Authorizing `identity:get_project(project_id=7b93d88c15094f44b25f8d85a7dbeb37)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:35.682170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec62f1-de94-4fc9-a209-db7c6e97e928 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:35.683865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 616/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jul 20 15:25:35 2022] GET /identity/v3/projects/7b93d88c15094f44b25f8d85a7dbeb37 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:25:35.690136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:35.785713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f06-35db-436f-bc9d-bb36190d7a40 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:35.796389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:35.819050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:35.819699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:35.820002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:35.820275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:35.821097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=7b93d88c15094f44b25f8d85a7dbeb37, user.id=1b134ac3f6f34892a62dcea74e026c33) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:25:35.821402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:35.822801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f829d4-6b33-4972-9daf-3e8f119df73d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:35.828451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 584/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:25:35 2022] GET /identity/v3/role_assignments?user.id=1b134ac3f6f34892a62dcea74e026c33&scope.project.id=7b93d88c15094f44b25f8d85a7dbeb37&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:25:45.674780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:45.825999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:30.798309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1c147-e296-41c3-8c40-1f1d4e318209 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.798309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1c147-e296-41c3-8c40-1f1d4e318209 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.798309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1c147-e296-41c3-8c40-1f1d4e318209 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.799443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 617/1201] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:30.805251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:30.805695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:30.805793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:30.825509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:30.826790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:30.916388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 585/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:30.941525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1c147-e296-41c3-8c40-1f1d4e318209 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:31.086732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1c147-e296-41c3-8c40-1f1d4e318209 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:31.129587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:31.130432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:31.130707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:31.130994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:31.142736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:31.144455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:31.166031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:31.167561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:31.169000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:31.169000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:31.169000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:31.169000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:31.172312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 618/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:31.173345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:31.331246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eab40-360d-44d4-84a9-47e2ee91f363 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:31.369972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:31.378242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:31.379265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:31.379729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:31.380207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:31.381115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:31.381605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:31.383206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8ea6f-20aa-4c2e-896d-b35cafa293f0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:31.400099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 586/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:31.406787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:31.484101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:31.552632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead578cf-cead-44a9-87cf-3cc493c55701 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:31.608478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:31.609176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:31.609520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:31.609872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:31.613764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:31.615517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb596-508f-4a0e-a602-e9451ae71e23 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:31.656388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 619/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jul 20 15:26:31 2022] POST /identity/v3/users => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:26:33.994517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83088335-4f83-4d4b-9eb9-6e37ea132f17 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:33.995192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83088335-4f83-4d4b-9eb9-6e37ea132f17 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:33.995192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83088335-4f83-4d4b-9eb9-6e37ea132f17 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:33.996759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 587/1206] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:26:34.002681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:34.003175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:34.003367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:34.021908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:34.023306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:34.111919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 620/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:34.129653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83088335-4f83-4d4b-9eb9-6e37ea132f17 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:34.257268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83088335-4f83-4d4b-9eb9-6e37ea132f17 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:34.298196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:34.299118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:34.299418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:34.299695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:34.312913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:34.322192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:34.323376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:34.324480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:34.327921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 588/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:26:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:34.329981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:34.378389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:34.448417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd82593-6aeb-449e-a2e1-10bef5aadfef None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:34.487228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:34.488213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:34.488213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:34.488498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:34.489252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:34.489560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:34.490936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:34.500626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 621/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:26:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:34.506316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:34.568712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:34.626974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e659c-bb4e-4daa-9e67-e013180ffa34 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:34.672444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:34.673273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:34.673565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:34.673835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:34.685298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:34.686801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:34.706160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:34.714555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:34.715653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:34.715653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:34.715653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:34.715653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:34.715653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 589/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:34.716031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:34.831668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:34.835012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af83a0-103b-46f5-a35a-d1a3ca07a943 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:34.877183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:34.877881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:34.878965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:34.878965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:34.879157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:34.879417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:34.880825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:34.893617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 622/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:26:34.899319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.016484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e9abf-ea5e-4073-ad34-84ea88a3a88b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.019056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:35.068052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.068858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.069137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.071791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.080212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.081667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:35.097266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:35.099919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:35.101153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 590/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:35.104591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.244407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b497dc9-9151-4d88-be1e-46fed00839ed None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.281997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.282598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.282852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.283127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.284302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:35.284616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.286342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.305000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 623/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:35.310671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.436389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ec512-8917-4c8d-8c1a-963a056cfa12 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.478825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c57856-580a-41fa-975c-60581901d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.479494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c57856-580a-41fa-975c-60581901d058 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.479795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c57856-580a-41fa-975c-60581901d058 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.480095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c57856-580a-41fa-975c-60581901d058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.519619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:35.565231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c57856-580a-41fa-975c-60581901d058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.567076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c57856-580a-41fa-975c-60581901d058 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.679405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 591/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:26:35.684938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.705453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:35.794502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab78a4-64f1-4c18-8e2e-9e9a297708be None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:35.833863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:35.834666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:35.834951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:35.835227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:35.846421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:35.847898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:35.866548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:35.868091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:35.869242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:35.869242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:35.869242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:35.869242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:35.869242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 624/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:35.877184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c57856-580a-41fa-975c-60581901d058 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:35.955920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:36.000678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c57856-580a-41fa-975c-60581901d058 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:36.036785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:36.038237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:36.038721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:36.039357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:36.040327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:36.040853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:36.042468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:36.055654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 592/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:36.061703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:36.145626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:36.188643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e279ec-c4d3-486f-8637-c1dd68c95ee5 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:36.226039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:36.227285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:36.227567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:36.227872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:36.236351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:36.237857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:36.253323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:36.254782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:36.255671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 625/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:36.263019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:36.387533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3ac6-bb0b-40b3-9573-047e4fff27e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:36.415109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:36.431360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:36.432142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:36.432529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:36.432768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:36.433815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:36.434082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:36.435585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe02ffe-4ba4-4bd1-a48f-04e505e9f017 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:36.452787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 593/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:36.458792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:36.574066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706661c-3270-406c-96b8-31a52e28c53d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:36.605888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:36.612514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:36.613174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:36.613561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:36.613867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:36.614836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=3bd6b54079a045dd9a526fafe8de8bbf) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:36.615147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:36.616821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3def0c5-4703-4f4d-92d0-8df18ea56e46 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:36.623724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 626/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:26:36 2022] GET /identity/v3/role_assignments?user.id=3bd6b54079a045dd9a526fafe8de8bbf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:26:38.728815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967198c-c641-42b9-abd9-55416ab71017 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:38.729353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967198c-c641-42b9-abd9-55416ab71017 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:38.729353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967198c-c641-42b9-abd9-55416ab71017 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:38.730908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 594/1220] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:38.736378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:38.736834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:38.737027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:38.754943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:38.756212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:38.827611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 627/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:38.852091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967198c-c641-42b9-abd9-55416ab71017 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:38.959840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967198c-c641-42b9-abd9-55416ab71017 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:38.996569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:38.997301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:38.997583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:38.997850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.008706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:39.016490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:39.017644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:39.018975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 595/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:26:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:39.024387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:39.139916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40156899-d826-4f90-b493-11bfae7eaa91 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:39.174191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:39.188342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:39.189024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:39.189311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:39.189588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.190348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] RBAC: Adding query filter params (name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:39.190659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:39.192086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:39.201330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 628/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:39.207250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:39.256335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:39.318295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef212796-ed46-4a14-8f54-b5b53e6051a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:39.355633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:39.356448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:39.356731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:39.357003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.370525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:39.372442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:39.392892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:39.393918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:39.395056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:39.395056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:39.395056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:39.395056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:39.395056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 596/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:39.400862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:39.509997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be54c0-51c6-4a52-ace1-b29ca52ba28f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:39.545799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:39.546504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:39.546787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:39.547063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.547833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:39.548170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:39.549621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:39.562756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 629/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:39.568699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:39.674877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:39.683693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d2ad-e992-4cbd-83fa-91d545ad1a96 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:39.719948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:39.720768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:39.721158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:39.721349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.729379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:39.730823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:39.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:39.746799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:39.747804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 597/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:39.754537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:39.801807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:39.866512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234548e-7fca-4d13-8d4f-4df87077d247 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:39.902374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:39.903050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:39.903331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:39.903605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:39.904486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:39.904795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:39.906215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:39.929957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 630/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:39.931111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.035025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e793-e0e8-4ad2-adba-5f5f248fdf5c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.074684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.075532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.075857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.076212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.088494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.090231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:40.110830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:40.111972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:40.112994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:40.112994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:40.112994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:40.112994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:40.115900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 598/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:26:40.119980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.236298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3d65a-0bac-45b4-84b0-b47515159776 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.238677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:40.276751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.277434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.277706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.277983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.278712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:40.279014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.280424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.297335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 631/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:40.298636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.308524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:40.409503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17951ac-fe9e-42f3-8674-a9142eea2e4a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.446764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.447595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.447915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.448194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.456217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.457710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:40.472736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:40.473690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:40.474619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 599/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:40.480438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.594382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe08eb-1196-489a-9e2b-dfd393a3b7ef None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.631863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.632549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.632829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.633101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.633886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:40.634201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.635657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3e061-999a-40bd-8cc5-99314d303efe None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.651246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 632/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:40.657433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:40.728129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:40.765289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc304386-5ef3-44aa-b1d8-37f17ef3c1ec None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:40.801390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:40.802285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:40.802571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:40.802845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/3bd6b54079a045dd9a526fafe8de8bbf/roles/c4686b68529845b1ab004a2eb7c25e4c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:40.818965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=3bd6b54079a045dd9a526fafe8de8bbf, role_id=c4686b68529845b1ab004a2eb7c25e4c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:40.820988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a818d-a6fb-422f-ab72-83310c1f98cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:40.833892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 600/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:26:40 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/3bd6b54079a045dd9a526fafe8de8bbf/roles/c4686b68529845b1ab004a2eb7c25e4c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:43.217484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf70f8-b17e-4d27-944c-a0b7b3b93624 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:43.218146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf70f8-b17e-4d27-944c-a0b7b3b93624 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:43.218146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf70f8-b17e-4d27-944c-a0b7b3b93624 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:43.219996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 633/1233] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:26:43.225395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:43.225829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:43.225956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:43.244056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:43.245341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:43.321146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 601/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:43.346311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf70f8-b17e-4d27-944c-a0b7b3b93624 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:43.378364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:43.478137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf70f8-b17e-4d27-944c-a0b7b3b93624 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:43.516838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:43.517642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:43.517927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:43.518287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:43.529138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:43.537246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:43.538248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:43.539697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 634/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jul 20 15:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:43.545615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:43.598415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:43.658325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87252b2f-8427-44b6-8b50-8798cd7c3b65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:43.700533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:43.701302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:43.701590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:43.701863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:43.704012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:43.704323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:43.705716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:43.716144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 602/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jul 20 15:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:43.721997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:43.817256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:43.839362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf90ade-fe5a-45c3-9b39-3217f488e84f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:43.880681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:43.881486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:43.881768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:43.882048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:43.893789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:43.895241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:43.913047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:43.914165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:43.915046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:43.915046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:43.915046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:43.915046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:43.915046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 635/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:26:43.920235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:44.026316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b6ef5-f75e-4a55-97ba-d4a5cf6ec527 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:44.037619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:44.067072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:44.067805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:44.068049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:44.068333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:44.069049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:44.069355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:44.071347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:44.083077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 603/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:44.088934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:44.198908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d135c1-32f6-4a37-b9b3-2ff79953a12f None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:44.238387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:44.239193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:44.239545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:44.239866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:44.243670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:44.250108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:44.251914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:44.268659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:44.269833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:44.270979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:44.271560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 636/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:44.277706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:44.385890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0674de-544a-4a22-8192-215fcd61367d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:44.429173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:44.430392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:44.430698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:44.430979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:44.431805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:44.432121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:44.433557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:44.452958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 604/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:26:44.460419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:44.583523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e593e-44d6-493e-8a55-bd707a40a1a1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:44.620857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:44.621620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:44.621907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:44.622185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:44.695019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:44.702235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:44.703703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:44.810695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 637/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:26:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:26:44.817492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:44.847263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:44.927229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9f86-d551-494b-ac72-204d841a2858 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:44.965462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:44.966208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:44.966493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:44.966768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:44.976940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:44.978474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:45.000216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:45.001365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:45.002252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:45.002252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:45.002252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:45.002252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:45.002252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 605/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:45.007781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:45.111704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:45.119192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e066-e85a-4be2-929c-c86c79f50ad0 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:45.161457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:45.162321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:45.162712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:45.162993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:45.164402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:45.164826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:45.166442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:45.182864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 638/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:26:45.190765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:45.282271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:45.306201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b70c-4b5f-4f6f-a6b4-fd3096236d2e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:45.342813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:45.343619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:45.343951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:45.344238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:45.351981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:45.353720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:45.370506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:45.371459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:45.372514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 606/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:45.377865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:45.518699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c39-8989-48c0-8dc7-0640654383e8 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:45.560916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:45.561511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:45.561802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:45.562060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:45.562870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:45.563916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:45.564737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9534770-75e5-471f-89f1-e611deb81da1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:45.581320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 639/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:26:45.588175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:45.702461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c85118-66b0-4d4b-b731-764682e7b760 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:45.740528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:45.741190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:45.741471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:45.741770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:45.742601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] RBAC: Adding query filter params (role.id=c4686b68529845b1ab004a2eb7c25e4c, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=3bd6b54079a045dd9a526fafe8de8bbf) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:45.742908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:45.744621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df969f-2178-4d9b-9b66-c0cd7e7a2855 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:45.751519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 607/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:26:45 2022] GET /identity/v3/role_assignments?user.id=3bd6b54079a045dd9a526fafe8de8bbf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=c4686b68529845b1ab004a2eb7c25e4c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:48.023029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71ed5-3d1a-471b-8c17-9d8c0ff61644 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.023657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71ed5-3d1a-471b-8c17-9d8c0ff61644 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.023740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71ed5-3d1a-471b-8c17-9d8c0ff61644 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.025457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 640/1247] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:48.031360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.031848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.032007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.037500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:48.050666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:48.051998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:48.130967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 608/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:48.159894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d71ed5-3d1a-471b-8c17-9d8c0ff61644 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:48.184302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:48.285229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d71ed5-3d1a-471b-8c17-9d8c0ff61644 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:48.324601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:48.325668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.325954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.326212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.336912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:48.344811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:48.345910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:48.346760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 641/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:48.353582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:48.472695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482fa2-3110-4ffb-9246-1c0aaa5098a5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:48.505428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:48.510948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:48.511667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.511973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.512219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.512970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:48.513280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:48.514645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:48.523777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 609/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:48.529518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:48.630216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:48.658415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49843450-5bc2-44ed-b695-600adc44c9da None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:48.697814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:48.698619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.698901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.699177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.711396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:48.713088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:48.732446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:48.733548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:48.734643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:48.734643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:48.734643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:48.734643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:48.734643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 642/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:48.739466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:48.844383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ced5e9-2be5-4754-8703-72dacefc04d0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:48.880996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:48.881597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:48.881880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:48.882172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:48.883430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:48.883741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:48.885147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:48.897390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 610/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:48.903224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.017346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d61-fc43-4cc4-a087-b67c4b0dccd1 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.044378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.054885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.056015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.056304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.056744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.066046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.067846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:49.084251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:49.085309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:49.086288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 643/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:49.091337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.158945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.213779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da701e3-24c8-4a8d-be90-2cedd0111868 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.252300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.252598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.252829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.253100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.254393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:49.254964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.256884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.278407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 611/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:49.285497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.397153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1c3a-1f2f-4c1a-9422-180bf01e7e3d None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.439749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.440449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.440734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.441013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.503509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.517090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.518578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.623721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 644/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:26:49.630672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.733593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2568d-79f7-4e89-a546-f7fc13bcef88 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.747834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.770621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.771419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.771871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.772146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.783308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.784877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:49.803387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:49.805026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:49.806066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:49.806066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:49.806066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:49.806066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:49.806066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 612/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:49.811523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:49.899306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.911940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a5db7-4630-42fa-b80d-e9d70716a688 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:49.946791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:49.947453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:49.947730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:49.948027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:49.948761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:49.949063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:49.950417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:49.962264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 645/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:49.967921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:50.063497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d6c1-5176-440e-b6d3-4a27d42bab31 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:50.095881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:50.096646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:50.096923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:50.097190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:50.103589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:50.105030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:50.118808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:50.120117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:50.121454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 613/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:50.125857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:50.230856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a517f6-cb2d-4d34-9b8d-83167294db78 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:50.264406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:50.265063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:50.265349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:50.265628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:50.266412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:50.266724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:50.268173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0266b0-afdd-414c-bffb-934b7faf68f7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:50.279905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 646/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:50.285945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:50.311549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:50.384231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6416d1-79cf-4801-8494-13b7952980b2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:50.417333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:50.417970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:50.418259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:50.418538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:50.419365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=3bd6b54079a045dd9a526fafe8de8bbf) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:50.419678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:50.421128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163197-317f-42b6-9e75-8ec764772ec8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:50.427816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 614/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:26:50 2022] GET /identity/v3/role_assignments?user.id=3bd6b54079a045dd9a526fafe8de8bbf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:52.452805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca456f1-2c34-4472-b01a-4d5ec0c39bab None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:52.453292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca456f1-2c34-4472-b01a-4d5ec0c39bab None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:52.453384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca456f1-2c34-4472-b01a-4d5ec0c39bab None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:52.454747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 647/1261] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:52.459698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:52.460087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:52.460822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:52.477985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:52.479172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:52.544215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 615/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:52.566095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca456f1-2c34-4472-b01a-4d5ec0c39bab None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:52.576807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:52.669933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca456f1-2c34-4472-b01a-4d5ec0c39bab None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:52.703083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:52.703829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:52.704115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:52.704388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:52.714779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:52.722349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:52.723432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:52.724628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 648/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:26:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:52.729696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:52.747723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:52.826356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1bf51-68b8-40f4-98dc-9852e69c06d6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:52.858993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:52.859646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:52.859954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:52.860230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:52.860970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:52.861277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:52.862624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:52.870734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 616/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:26:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:52.876385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:52.941451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:52.974626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f22d-17ea-49a9-a0be-1498d4a74b89 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.006512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.007828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.008111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.008389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.018568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.020041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:53.037157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:53.038169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:53.039429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:53.039429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:53.039429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:53.039429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:53.039429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 649/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:53.044312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.123206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:53.142383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c5f6-e971-49ce-bae6-075e302ef6c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.184543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.185488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.185898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.186688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.188209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:53.188673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.191409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.213569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 617/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:53.219507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.316846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1140715-06c4-42d7-933b-7931036ed554 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.336018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:53.349714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.350502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.350778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.351048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.358319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.359736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:53.374269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:53.375445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:53.376668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 650/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:53.381330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.478119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7737-3572-434c-827e-8e707f94d569 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.511102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.511785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.512073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.512344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.513179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:53.513486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.514898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.523207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:53.531436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 618/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:53.537129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.637645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e4f7-562b-4d28-9157-3a62ac6ecdae None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.669840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.670614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.670895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.671170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.681944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.683381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:53.700065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:53.701081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:53.702347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:53.702347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:53.702347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:53.702347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:53.702347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 651/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:53.706978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.804090 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f367a0e-35ba-4b17-9d8b-e7d6e474092c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.836842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.837486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.837770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.838046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.838758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:53.839064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.840454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:53.851911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 619/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:53.857423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:53.888170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:53.954413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d085d-f2e4-4234-b67f-30325ac06f86 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:53.987151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:53.987947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:53.988230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:53.988506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:53.995211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:53.996660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:54.010432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:54.011466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:54.012472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 652/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:26:54.017225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:54.073975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:54.115133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2800c1-4b6e-4953-b450-f7447c44bdfe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:54.148499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:54.149105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:54.149389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:54.149671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:54.150458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:54.150763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:54.152222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a24c8-3b3c-4d3e-bde7-ab7c89211f14 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:54.165707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 620/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:54.172065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:54.261811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:54.270887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519580a7-6910-443c-95ce-1390f50bfc24 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:54.304186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:54.305053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:54.305331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:54.305603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] PATH_INFO: `/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/3bd6b54079a045dd9a526fafe8de8bbf/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:54.307513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] RBAC: Authorizing `identity:create_grant(project_id=f24b63df63c344e68a24f6f2876f92e5, user_id=3bd6b54079a045dd9a526fafe8de8bbf, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:54.308971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5914d1-711b-4b87-b18e-69554a20c110 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:54.321285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 653/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jul 20 15:26:54 2022] PUT /identity/v3/projects/f24b63df63c344e68a24f6f2876f92e5/users/3bd6b54079a045dd9a526fafe8de8bbf/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:26:56.296707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeb1b5-a805-4f1b-a424-0831374c9256 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:56.297233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeb1b5-a805-4f1b-a424-0831374c9256 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:56.297233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deeb1b5-a805-4f1b-a424-0831374c9256 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:56.298610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 621/1274] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:56.303629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:56.304536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:56.304754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:56.321444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:56.322596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:56.386579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 654/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:26:56.409083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deeb1b5-a805-4f1b-a424-0831374c9256 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:56.511298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deeb1b5-a805-4f1b-a424-0831374c9256 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:56.527953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:56.544915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:56.545702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:56.545987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:56.546262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:56.556792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:26:56.564428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:56.565483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:56.566516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 622/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jul 20 15:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:56.571833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:56.649243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:56.667976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775a78b-cd56-4156-be47-20d1c75208c4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:56.700796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:56.701453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:56.701735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:56.702012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:56.702757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:56.703063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:56.704451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:56.712290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 655/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jul 20 15:26:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:56.717828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:56.815944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fceef-fc53-47c9-bee0-a331ebcceb3b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:56.848288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:56.849060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:56.849343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:56.849615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:56.860452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:56.861887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:56.878869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:56.879930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:56.880957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:56.880957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:56.880957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:56.880957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:56.880957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 623/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:56.886065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:56.984091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b40ab-c268-493a-90ba-bfdd7b96bbb3 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.017085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.017751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.018037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.018317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.019041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:57.019350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.020748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.032781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 656/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:57.038891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:57.068578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:57.136740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e768d3-29c1-43e6-a3c6-822399df9fd5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.175574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.176568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.176882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.177289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.185746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.187216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:26:57.202011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:57.203504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:26:57.205019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 624/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:57.209339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:57.214918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:57.309084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6debc6-429d-4f8d-948d-fb6bc8651cc7 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.342904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.343564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.343874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.344156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.344992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:57.345306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.346752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.363167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 657/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:57.368911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:57.458583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:57.468748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ab388-f821-4c6f-8b2e-683dba9992a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.500934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.501578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.501863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.502138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.577368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.578803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.664631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:57.674730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 625/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:26:57.681013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:57.740938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:57.779178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f219f9-bda4-42ac-959e-0d8f4cbedddf None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.812049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.812822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.813104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.813377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.823710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.825183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:26:57.843096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:57.844285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:57.845572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:26:57.845572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:26:57.845572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:26:57.845572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:57.845572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 658/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:57.849970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:57.947057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b504c-da62-4cb6-a017-15635f602287 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:57.980236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:57.980889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:57.981174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:57.981451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:57.982170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:57.982478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:57.983860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:57.995145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 626/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:58.000680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:58.096518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6dc8-3bde-4198-a117-cb3403c2d808 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:58.110233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:58.129901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:58.130672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:58.130953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:58.131225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:58.138240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:58.139663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:26:58.153587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application with Lock( Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:26:58.154625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: ERROR keystone.server.flask.application Jul 20 15:26:58.155899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 659/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jul 20 15:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:26:58.160448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:58.209347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:58.258452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f28a7a-2d91-4f4a-bfd9-02fbfe2c319e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:58.290749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:58.291403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:58.291687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:58.291990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:58.292774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:58.293084 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:58.294513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cafbcc-202f-4546-92c4-4c11b73204a0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:58.307352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 627/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jul 20 15:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:26:58.313640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:58.411958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dda12-0f60-4077-8826-972dc9300524 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:58.444855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:58.445507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:58.445788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:58.446062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:58.446891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] RBAC: Adding query filter params (role.id=d9ec2045400649bdb6f718caf2f62488, scope.project.id=f24b63df63c344e68a24f6f2876f92e5, user.id=3bd6b54079a045dd9a526fafe8de8bbf) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:26:58.447231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:58.448677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd2430-492c-449c-89e6-c52d04c5d5e5 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:58.454365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 660/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jul 20 15:26:58 2022] GET /identity/v3/role_assignments?user.id=3bd6b54079a045dd9a526fafe8de8bbf&scope.project.id=f24b63df63c344e68a24f6f2876f92e5&role.id=d9ec2045400649bdb6f718caf2f62488 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:00.489978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758604c2-1139-43c1-a553-8769eb4528a0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:00.490450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758604c2-1139-43c1-a553-8769eb4528a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:00.490548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758604c2-1139-43c1-a553-8769eb4528a0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:00.492523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 628/1288] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:27:00.497783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:00.498235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:00.498336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:00.516660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:00.517830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:00.547138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:00.584184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 661/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:27:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:27:00.607025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758604c2-1139-43c1-a553-8769eb4528a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:00.705229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758604c2-1139-43c1-a553-8769eb4528a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:00.716000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:00.738806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:00.739589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:00.739913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:00.740190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:00.740842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:00.742710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:27:00.754569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:27:00.755834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:00.756901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 629/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:27:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:27:00.762259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:00.856098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b5e28-1f9f-4161-8046-4e22194ea211 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:00.889580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:00.890223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:00.890502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:00.890772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:00.891466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:00.891783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:00.893145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe69af-0a56-4eab-b395-5c0893737c56 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:00.897822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 662/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jul 20 15:27:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:00.903362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:01.001829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b6d3-6d12-4d61-b5a6-cb311afa975e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:01.039145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:01.039841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:01.040129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:01.040405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:01.041109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:01.041420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:01.042786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ff951-13de-42bb-8847-b7b972d4e496 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:01.047332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 630/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jul 20 15:27:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:03.082852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9aeed-de7d-44c8-a08a-ab57a756e3b2 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:03.083382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9aeed-de7d-44c8-a08a-ab57a756e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:03.083382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9aeed-de7d-44c8-a08a-ab57a756e3b2 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:03.084798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 663/1293] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:27:03.090500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2951-98c3-4d70-a070-cd114d83d9d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:03.090930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2951-98c3-4d70-a070-cd114d83d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:03.091051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed2951-98c3-4d70-a070-cd114d83d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:03.108147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-98ed2951-98c3-4d70-a070-cd114d83d9d1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:03.109330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-98ed2951-98c3-4d70-a070-cd114d83d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:03.138058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:03.177907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 631/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:27:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:27:03.201403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9aeed-de7d-44c8-a08a-ab57a756e3b2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:03.233678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:03.303848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9aeed-de7d-44c8-a08a-ab57a756e3b2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:03.337804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:03.338396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:03.338675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:03.338949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:03.339668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:03.341065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99e98-ae2a-4d3c-a3c8-e51443cddc0a None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:03.353270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 664/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jul 20 15:27:03 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:27:05.363850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7f9b-1d00-4451-a075-abf7f9a5dd82 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:05.364405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7f9b-1d00-4451-a075-abf7f9a5dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:05.364405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7f9b-1d00-4451-a075-abf7f9a5dd82 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:05.365812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 632/1296] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:27:05.371638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:05.372117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:05.372258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:05.388511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:05.389664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:05.463044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 665/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:27:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:27:05.485682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7f9b-1d00-4451-a075-abf7f9a5dd82 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:05.583666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7f9b-1d00-4451-a075-abf7f9a5dd82 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:05.623238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:05.623977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:05.624261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:05.624533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:05.625185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:05.626553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:27:05.638134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:27:05.639276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:05.640348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 633/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:27:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:27:05.645301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:05.662913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:05.741845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ff8a4-0832-4a02-ae00-3f791732c7ca None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:05.774515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:05.775156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:05.775433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:05.775704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:05.776555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:05.776858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:05.778212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:05.783155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 666/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jul 20 15:27:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:05.789455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:05.820189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:05.886866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5009b-a534-4ce5-8ce9-986e547b7c02 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:05.919551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:05.920229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:05.920515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:05.920792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:05.921574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] RBAC: Adding query filter params (interface=public, service_id=2cf36d061fa44ebab3125ce9ac47b17a, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:05.921884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:05.923234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074da50-b750-4bcc-9844-9a14f751a24c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:05.928183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 634/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jul 20 15:27:05 2022] GET /identity/v3/endpoints?service_id=2cf36d061fa44ebab3125ce9ac47b17a&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:05.933724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:06.013038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:06.031148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf5f9-8b6f-4bba-9478-a13c9acb4e42 None admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:06.063891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:06.064540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:06.064825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:06.065103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:06.065786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:06.067140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df9023-54d1-4f19-93dc-85b50cb622d7 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:06.073794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 667/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jul 20 15:27:05 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:27:08.094923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0353-8589-400f-b2c0-d5d2653d13e0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:08.095475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0353-8589-400f-b2c0-d5d2653d13e0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:08.095475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0353-8589-400f-b2c0-d5d2653d13e0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:08.096995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 635/1302] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:27:08.102514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:08.102926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:08.103036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:08.119907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:08.121057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:08.192227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 668/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:27:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:27:08.215212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0353-8589-400f-b2c0-d5d2653d13e0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:08.314272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:08.322224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0353-8589-400f-b2c0-d5d2653d13e0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:08.354775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:08.355514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:08.355824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:08.356106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:08.356756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:08.358124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:27:08.370632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:27:08.371745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jul 20 15:27:08.376954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 636/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jul 20 15:27:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:27:08.377935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:08.451769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:08.473873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ddcd-6e5c-4d2c-83e3-c688dd5fb176 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:08.507595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:08.508279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:08.508558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:08.508830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] PATH_INFO: `/v3/services` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:08.509527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:08.509833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:08.511193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d419f2-e3f7-4a29-a958-6217958bd855 None admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:08.516233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 669/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jul 20 15:27:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:08.522005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:08.622774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1948d-f20c-4e65-a309-66b93f78184c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:08.656075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:08.656726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:08.657010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:08.657288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:08.658027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:08.659381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eae98b-25b3-4a35-8bd2-25fad5370391 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:08.676900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 637/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jul 20 15:27:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:27:18.513238 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=30 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:18.672626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:11.146620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ec4a-a13a-4dca-9e4b-d59c885a0da4 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:11.147807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ec4a-a13a-4dca-9e4b-d59c885a0da4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:11.148336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ec4a-a13a-4dca-9e4b-d59c885a0da4 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:11.153609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 670/1307] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:11.155372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07692b0-a60e-4959-913d-cd939260ed1b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:11.156289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07692b0-a60e-4959-913d-cd939260ed1b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:11.156793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07692b0-a60e-4959-913d-cd939260ed1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:11.255930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f07692b0-a60e-4959-913d-cd939260ed1b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:11.258328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f07692b0-a60e-4959-913d-cd939260ed1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:11.420620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 638/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:11.467408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:11.467846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:11.468317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:11.470047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 671/1309] 10.222.0.27 () {54 vars in 965 bytes} [Wed Jul 20 15:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:11.478110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812fc60-05ed-4af6-8ade-625fdef27fb9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:11.479176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812fc60-05ed-4af6-8ade-625fdef27fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:11.479610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812fc60-05ed-4af6-8ade-625fdef27fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:11.669167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9812fc60-05ed-4af6-8ade-625fdef27fb9 None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:11.670301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9812fc60-05ed-4af6-8ade-625fdef27fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:11.703896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:11.766577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 639/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Jul 20 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:11.776038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:11.778011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:11.893900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f462-3c87-4c4b-9f75-3c0226bf8843 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:11.976336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:11.976925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:11.977225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:11.977484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:11.979128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:12.037695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:12.130550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:12.132592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa78dd-ed92-4d59-84c2-8ce2dcf8177c service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:12.159821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 672/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:14.574336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:14.574867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:14.574867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:14.576339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 640/1312] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:14.581882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5950-e9cd-4092-a60e-133dd605fbdb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:14.582322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5950-e9cd-4092-a60e-133dd605fbdb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:14.582577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce5950-e9cd-4092-a60e-133dd605fbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:14.656780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-63ce5950-e9cd-4092-a60e-133dd605fbdb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:14.658016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-63ce5950-e9cd-4092-a60e-133dd605fbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:14.762993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 673/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:14.801753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:14.803204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:14.904381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1bdb1-62c4-4198-aadc-cdde0c986f1e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:14.976954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:14.979340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:14.980038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:14.980661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:14.981141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:14.983256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:15.094577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:15.096029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa1a4d-8e9f-48af-97fa-c8b1025cd4a6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:15.115432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 641/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:17.458996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.459537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.460162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.461530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 674/1315] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:17.466780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8953-fb02-4d01-a035-b4738a05004f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.467174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8953-fb02-4d01-a035-b4738a05004f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.467335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f8953-fb02-4d01-a035-b4738a05004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.531359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9f4f8953-fb02-4d01-a035-b4738a05004f None None] MFA Rules not processed for user `7c6356db0d524b36937100ddefe10ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:17.532607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9f4f8953-fb02-4d01-a035-b4738a05004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.583991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:17.628011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 642/1316] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:17.667869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:17.669387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.679192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:17.773458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d2be4-14e0-4d82-8e68-961bd519518d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:17.844356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:17.844999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:17.845280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:17.845553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:17.847171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:17.907548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:17.954053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:17.955411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f3653-5bef-45c8-b0d8-306077aea40c service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:17.972808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 675/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:20.624188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60b75e-e1fc-441b-a3b9-09ddd7ce4de5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.624715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60b75e-e1fc-441b-a3b9-09ddd7ce4de5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.624715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60b75e-e1fc-441b-a3b9-09ddd7ce4de5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.626126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 643/1318] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:20.631389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3848-8395-4114-a924-19653c503442 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.631824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3848-8395-4114-a924-19653c503442 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.632000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3848-8395-4114-a924-19653c503442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.695914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-816e3848-8395-4114-a924-19653c503442 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:20.697048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-816e3848-8395-4114-a924-19653c503442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:20.793013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 676/1319] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:20.843789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.844083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.844365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.845766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 644/1320] 10.222.0.27 () {54 vars in 965 bytes} [Wed Jul 20 15:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:20.852034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380eea9-4aa5-4b07-b6b2-bc13e0540289 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:20.852326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380eea9-4aa5-4b07-b6b2-bc13e0540289 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:20.852610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380eea9-4aa5-4b07-b6b2-bc13e0540289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:20.873202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:20.926438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b380eea9-4aa5-4b07-b6b2-bc13e0540289 None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:20.927624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b380eea9-4aa5-4b07-b6b2-bc13e0540289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:21.002273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 677/1321] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Jul 20 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:21.011039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:21.012581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:21.114558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd839-ad8b-4b1b-8eea-88cfdeea3cb9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:21.186385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:21.187028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:21.187306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:21.187572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:21.189219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:21.197525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:21.297246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:21.298588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348139f-68a4-4792-9fcf-490eaee0f489 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:21.316432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 645/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:24.496722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.497248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.497248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.498637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 678/1323] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:24.503915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4edc-2655-4b67-9e14-5c2549162736 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.504331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4edc-2655-4b67-9e14-5c2549162736 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.504481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4edc-2655-4b67-9e14-5c2549162736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.569615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ca9e4edc-2655-4b67-9e14-5c2549162736 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:24.570837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca9e4edc-2655-4b67-9e14-5c2549162736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:24.635590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:24.679395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 646/1324] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:24.716293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:24.717822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:24.826460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7de5-f71a-441f-9cfc-50ad0eb1c5d9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:24.839893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:24.902338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:24.902988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:24.903270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:24.903545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:24.905212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:25.013736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:25.015090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddf7d6-585f-458d-8b7c-6a3c54c0ceb3 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:25.033622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 679/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:27.113096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:27.113618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:27.113618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:27.114987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 647/1326] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:27.120546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f71-99a5-458d-b735-707369f7c0bc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:27.120978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f71-99a5-458d-b735-707369f7c0bc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:27.121235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f71-99a5-458d-b735-707369f7c0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:27.169082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:27.187302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-968e2f71-99a5-458d-b735-707369f7c0bc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:27.188528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-968e2f71-99a5-458d-b735-707369f7c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:27.285246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 680/1327] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:27.321529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:27.323056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:27.431792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7386b-a71e-4a12-8f2e-e0de81364cea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:27.510395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:27.511060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:27.511343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:27.511618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:27.513395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:27.532459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:27.627257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:27.628715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e19cf9-0216-4703-b1be-74082dc11600 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:27.647403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 648/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:29.721275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:29.721779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:29.721888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:29.723273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 681/1329] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:29.728663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8508-9332-494f-951d-c3b875617811 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:29.729093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8508-9332-494f-951d-c3b875617811 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:29.729213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c8508-9332-494f-951d-c3b875617811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:29.794602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-347c8508-9332-494f-951d-c3b875617811 None None] MFA Rules not processed for user `7c6356db0d524b36937100ddefe10ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:29.795930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-347c8508-9332-494f-951d-c3b875617811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:29.868653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:29.902601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 649/1330] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:29.942918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:29.945095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:30.060924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a32cad-1948-451b-90e8-63e7bbffa163 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:30.080915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:30.136795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:30.137305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:30.137592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:30.137866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:30.139506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:30.247126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:30.249756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e17235-9b5e-4b8b-8b9b-27444ee16c67 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:30.267066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 682/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:32.572793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:32.573362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:32.573362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:32.574709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 650/1332] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:32.579970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05996-a808-44a6-ba34-6de2694ab2e1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:32.580407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05996-a808-44a6-ba34-6de2694ab2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:32.580534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05996-a808-44a6-ba34-6de2694ab2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:32.641736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:32.644615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c6e05996-a808-44a6-ba34-6de2694ab2e1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:32.646401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c6e05996-a808-44a6-ba34-6de2694ab2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:32.740284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 683/1333] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:32.778919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:32.780502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:32.882871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b107a8d-4fb9-4179-9a9c-0013c5a56cee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:32.953834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:32.954408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:32.954723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:32.955024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:32.956667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:32.983575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:33.066297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:33.067797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94f60e-27c5-4c88-9a16-135871b51092 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:33.086130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 651/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:35.160931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:35.161480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:35.161480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:35.163347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 684/1335] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:35.168725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609ec1-9647-4734-8bae-9abfb3d7069a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:35.169098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609ec1-9647-4734-8bae-9abfb3d7069a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:35.169280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609ec1-9647-4734-8bae-9abfb3d7069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:35.232323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9a609ec1-9647-4734-8bae-9abfb3d7069a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:35.233622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a609ec1-9647-4734-8bae-9abfb3d7069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:35.309069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:35.330898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 652/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:35.371780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:35.374669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:35.481796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122183f3-c40c-40aa-a4ed-d5a3d41786ce None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:35.509689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:35.554314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:35.554966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:35.555247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:35.555516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:35.557195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:35.664532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:35.665961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ccb62-4b61-41e5-b517-ca0dfb4361f8 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:35.683375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 685/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:38.044903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:38.045461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:38.045461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:38.046812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 653/1338] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:38.052135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702fe1-b7bf-40b2-8c63-716ef4189678 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:38.052570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702fe1-b7bf-40b2-8c63-716ef4189678 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:38.052867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702fe1-b7bf-40b2-8c63-716ef4189678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:38.117424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0d702fe1-b7bf-40b2-8c63-716ef4189678 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:38.118727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0d702fe1-b7bf-40b2-8c63-716ef4189678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:38.129321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:38.217424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 686/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:38.254667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:38.256238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:38.361165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1e0e8-4343-4ce5-9fbd-c4845dbdb4a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:38.433421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:38.434069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:38.434349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:38.434622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:38.436272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:38.470873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:38.543723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:38.545241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cae937-5b5d-4903-8ef2-54edba34e8e1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:38.562890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 654/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:42.601916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:42.602465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:42.602465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:42.603881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 687/1341] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:42.609066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061a1d-ee9d-454b-999c-3efe3ae2818f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:42.609480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061a1d-ee9d-454b-999c-3efe3ae2818f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:42.609646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061a1d-ee9d-454b-999c-3efe3ae2818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:42.675099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-94061a1d-ee9d-454b-999c-3efe3ae2818f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:42.676313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94061a1d-ee9d-454b-999c-3efe3ae2818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:42.763273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:42.774774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 655/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:28:42.814411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:42.815991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:42.924866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff6675-6c67-4c09-94b4-6fe800bac2d7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:42.964062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:43.000680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:43.001335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:43.001619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:43.001897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:43.003527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:43.114944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:43.116454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcaf59-4c83-4253-b2ce-4a0483dd50ad service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:43.135038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 688/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:45.830381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.830941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.830941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.832380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 656/1344] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:45.838655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b912fa8-e421-4d61-98e3-94217fe14f1b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:45.839148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b912fa8-e421-4d61-98e3-94217fe14f1b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:45.839306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b912fa8-e421-4d61-98e3-94217fe14f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:45.907666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3b912fa8-e421-4d61-98e3-94217fe14f1b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:45.908895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3b912fa8-e421-4d61-98e3-94217fe14f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:45.926863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:46.011993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 689/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:46.053421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:46.055061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.169572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c40776-072a-4b1c-a059-dd432c383f1d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:46.251949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:46.252609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:46.252895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:46.253172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:46.254825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:46.302130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:46.376467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:46.377903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317b50a-0349-4eb8-82cc-e75386273149 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:46.395587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 657/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:50.471570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:50.472161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:50.472161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:50.473542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 690/1347] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:50.478762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140dcd-964f-468f-8629-7d5589534666 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:50.479194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140dcd-964f-468f-8629-7d5589534666 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:50.479315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140dcd-964f-468f-8629-7d5589534666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:50.547265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-68140dcd-964f-468f-8629-7d5589534666 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:50.548440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-68140dcd-964f-468f-8629-7d5589534666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:50.664372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:50.667940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 658/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:28:50.706735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:28:50.708384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:50.831900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84220c1-efd9-45be-a338-09bed924b8a6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:28:50.883377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:50.910085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:28:50.910747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:50.911029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:50.911304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:50.913520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:51.027428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:28:51.028934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd15f9e-6afc-43dc-947e-d7988d8ae136 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:28:51.048756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 691/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:28:55.969088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5e38-0746-48fd-94c0-a8a6b4fc2f9e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:55.969582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5e38-0746-48fd-94c0-a8a6b4fc2f9e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:55.969665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5e38-0746-48fd-94c0-a8a6b4fc2f9e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:55.971032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 659/1350] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:28:55.976580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8f575-7b59-44de-8e63-a0927f9a67f5 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:28:55.977009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8f575-7b59-44de-8e63-a0927f9a67f5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:28:55.977123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8f575-7b59-44de-8e63-a0927f9a67f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:28:56.117905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-03d8f575-7b59-44de-8e63-a0927f9a67f5 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:28:56.119087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-03d8f575-7b59-44de-8e63-a0927f9a67f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:28:56.138492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:56.235969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 692/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:00.354401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae160e-8302-4e59-b1b6-507869fcdeaa None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:00.354912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae160e-8302-4e59-b1b6-507869fcdeaa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:00.354912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae160e-8302-4e59-b1b6-507869fcdeaa None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:00.356468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 660/1352] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:00.361775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae1ec-5717-44a1-b1da-3b17d1070c61 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:00.363131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae1ec-5717-44a1-b1da-3b17d1070c61 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:00.363660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae1ec-5717-44a1-b1da-3b17d1070c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:00.456706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-78eae1ec-5717-44a1-b1da-3b17d1070c61 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:00.457921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-78eae1ec-5717-44a1-b1da-3b17d1070c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:00.541737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:00.586431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 693/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:01.995945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbaa-e399-4199-90a8-8bb8c0c9631b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:01.996558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbaa-e399-4199-90a8-8bb8c0c9631b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:01.996558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbaa-e399-4199-90a8-8bb8c0c9631b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:01.997848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 661/1354] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:02.002868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090c1-1a7f-488c-8304-93ad73d2f97a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:02.003301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090c1-1a7f-488c-8304-93ad73d2f97a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:02.003438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090c1-1a7f-488c-8304-93ad73d2f97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:02.077367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1fe090c1-1a7f-488c-8304-93ad73d2f97a None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:02.079412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1fe090c1-1a7f-488c-8304-93ad73d2f97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:02.189078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 694/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:05.592846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cece-a235-48d3-949d-7fb162fa02b1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:05.593727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cece-a235-48d3-949d-7fb162fa02b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:05.594239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cece-a235-48d3-949d-7fb162fa02b1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:05.600558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 662/1356] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:05.601793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510dd9-2ea2-4c7e-a16c-d3fb4e4cbbc0 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:05.602215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510dd9-2ea2-4c7e-a16c-d3fb4e4cbbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:05.602396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68510dd9-2ea2-4c7e-a16c-d3fb4e4cbbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:05.647943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:05.682724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-68510dd9-2ea2-4c7e-a16c-d3fb4e4cbbc0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:05.683851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-68510dd9-2ea2-4c7e-a16c-d3fb4e4cbbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:05.786665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 695/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:05.843195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:05.844176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:05.844848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:05.847666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 663/1358] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jul 20 15:29:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:05.858287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84a42-607d-4cbe-9651-51b37d1212cf None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:05.858570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84a42-607d-4cbe-9651-51b37d1212cf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:05.858945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f84a42-607d-4cbe-9651-51b37d1212cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:05.947809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b0f84a42-607d-4cbe-9651-51b37d1212cf None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:05.949379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b0f84a42-607d-4cbe-9651-51b37d1212cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:06.004137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:06.081263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 696/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jul 20 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:06.088875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:06.090637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:06.246584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc9e6-2d1e-4227-a30a-b75e665d3642 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:06.328006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:06.373841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:06.374497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:06.374775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:06.375047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:06.377217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:06.483189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:06.484822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119fcf9-f803-4808-92de-33efb9b91b9d service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:06.507566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 664/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:08.772513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:08.773063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:08.773063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:08.774645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 697/1361] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:08.779634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5a654-7b09-4cfe-96d0-d92046424d2f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:08.780098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5a654-7b09-4cfe-96d0-d92046424d2f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:08.780223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5a654-7b09-4cfe-96d0-d92046424d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:08.849331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-47c5a654-7b09-4cfe-96d0-d92046424d2f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:08.850536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47c5a654-7b09-4cfe-96d0-d92046424d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:08.891617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:08.971544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 665/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:09.020366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:09.022424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:09.234530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22a70b-6411-43dd-a32d-046ffa8ae623 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:09.246609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:09.361388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:09.362084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:09.362481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:09.362760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:09.364444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:09.461954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:09.464062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d124ca-9c73-4dbe-baff-0f440e66e8aa service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:09.484062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 698/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:11.883915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbcfbb-d0ab-4757-b9b5-fb03eecf8bc1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:11.884474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbcfbb-d0ab-4757-b9b5-fb03eecf8bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:11.884474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbcfbb-d0ab-4757-b9b5-fb03eecf8bc1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:11.885840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 666/1364] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:29:11.891430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a9a2-d4c8-4792-a695-49f43847ba77 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:11.891883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a9a2-d4c8-4792-a695-49f43847ba77 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:11.892051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a9a2-d4c8-4792-a695-49f43847ba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:11.908046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:11.963025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-acf8a9a2-d4c8-4792-a695-49f43847ba77 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:11.964260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-acf8a9a2-d4c8-4792-a695-49f43847ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:12.067081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 699/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:12.118121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:12.119905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:12.120524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:12.122206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 667/1366] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jul 20 15:29:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:12.127938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ee01-4ef6-4da6-9f51-2696b72b506a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:12.128296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ee01-4ef6-4da6-9f51-2696b72b506a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:12.128635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ee01-4ef6-4da6-9f51-2696b72b506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:12.208815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-aff0ee01-4ef6-4da6-9f51-2696b72b506a None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:12.210011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aff0ee01-4ef6-4da6-9f51-2696b72b506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:12.247963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:12.341221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 700/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jul 20 15:29:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:12.349001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:12.350839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:12.523578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a68a5-27c5-47da-86ff-7693489e5809 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:12.545316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:12.653013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:12.653669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:12.653945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:12.654213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:12.655860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:12.756676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:12.758282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9699c9-836d-4381-b71f-cda2f4dec3c8 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:12.782369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 668/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:13.739662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0c50c-a58e-4a42-a8c6-0ec736a50bbf None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:13.740222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0c50c-a58e-4a42-a8c6-0ec736a50bbf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:13.740222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0c50c-a58e-4a42-a8c6-0ec736a50bbf None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:13.741709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 701/1369] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:13.746877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0a358-852a-4323-bfc6-e30cbaa6245a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:13.747283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0a358-852a-4323-bfc6-e30cbaa6245a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:13.747427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0a358-852a-4323-bfc6-e30cbaa6245a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:13.779582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:13.815377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b5e0a358-852a-4323-bfc6-e30cbaa6245a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:13.816597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b5e0a358-852a-4323-bfc6-e30cbaa6245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:13.917983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 669/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:14.262213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:14.262213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:14.262213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:14.264586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 702/1371] 10.222.0.27 () {54 vars in 964 bytes} [Wed Jul 20 15:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:14.268626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ac64-af06-4c42-afc5-a82fe944602d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:14.268912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ac64-af06-4c42-afc5-a82fe944602d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:14.269184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ac64-af06-4c42-afc5-a82fe944602d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:14.348112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66c6ac64-af06-4c42-afc5-a82fe944602d None None] MFA Rules not processed for user `51875df6e7b1444cb31e793aa8727363`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:14.349834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66c6ac64-af06-4c42-afc5-a82fe944602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:14.398528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:14.439085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 670/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Jul 20 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:14.440614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:14.442469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:14.558187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c452b26-9415-4c5f-86c2-037a1929db22 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:14.601149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:14.637623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:14.638284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:14.638566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:14.638841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:14.640504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:14.761580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:14.763029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a55410-8448-4382-87c6-6a377c695764 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:14.782810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 703/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:14.892698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:14.893199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:14.893199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:14.894579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 671/1374] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:14.900217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ff58e-a8f2-4ac8-b977-5d136f41b018 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:14.900554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ff58e-a8f2-4ac8-b977-5d136f41b018 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:14.900859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ff58e-a8f2-4ac8-b977-5d136f41b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:14.965589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e46ff58e-a8f2-4ac8-b977-5d136f41b018 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:14.966794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e46ff58e-a8f2-4ac8-b977-5d136f41b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:14.988285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:15.080356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 704/1375] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:15.113799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:15.116145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:15.281387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c68e6c-b303-4993-8d0d-1e7f31035e4a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:15.334066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:15.399092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:15.399218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:15.399480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:15.399752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:15.401438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:15.500014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:15.501622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc252a-a337-4878-b6f1-209a6d7cfe95 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:15.520517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 672/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:29:17.605433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:17.605946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:17.606042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:17.607344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 705/1377] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:17.612476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6928-9d66-42c7-8c3e-32f98efbb63a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:17.612883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6928-9d66-42c7-8c3e-32f98efbb63a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:17.613039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6928-9d66-42c7-8c3e-32f98efbb63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:17.682890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-83cd6928-9d66-42c7-8c3e-32f98efbb63a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:17.684083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-83cd6928-9d66-42c7-8c3e-32f98efbb63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:17.691081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:17.809645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 673/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:17.854264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:17.855916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:18.002783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc2a8f-2cec-46a9-9a97-11cfebb8853a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:18.006876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:18.181523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:18.182271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:18.182626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:18.182997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:18.184723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:18.312487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:18.314989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b4a69-05f8-47f9-bf13-06fe00a7e883 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:18.341187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 706/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:20.429949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:20.430473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:20.430473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:20.435548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 674/1380] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:20.437208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7373f2-0c01-470b-960d-d6346902ab23 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:20.437638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7373f2-0c01-470b-960d-d6346902ab23 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:20.437781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7373f2-0c01-470b-960d-d6346902ab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:20.448834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:20.510031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-fa7373f2-0c01-470b-960d-d6346902ab23 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:20.511462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fa7373f2-0c01-470b-960d-d6346902ab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:20.632387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 707/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:20.674020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:20.675786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:20.827296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:20.839512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9abc-0c09-4afa-8199-14ccfaee0e68 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:20.964810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:20.965460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:20.965737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:20.966011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:20.967876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:21.066406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:21.068805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242210b-18fe-4a00-ae1b-4ebd6e76a8b2 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:21.081479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:21.091479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 675/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:23.529227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:23.529756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:23.529869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:23.531298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 708/1383] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:23.538000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161dea-b4c0-4fea-a630-e3002b5be512 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:23.538408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161dea-b4c0-4fea-a630-e3002b5be512 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:23.538748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe161dea-b4c0-4fea-a630-e3002b5be512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:23.617709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fe161dea-b4c0-4fea-a630-e3002b5be512 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:23.619008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe161dea-b4c0-4fea-a630-e3002b5be512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:23.718569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 676/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:23.762763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:23.764369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:23.864438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:23.941603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0c510-eb3d-48e7-a62f-59a9b908ebd1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:24.077815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:24.078391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:24.078683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:24.078959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:24.080641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:24.145601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:24.193783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:24.195923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e80c1-4dd6-4d62-b712-9f6e2ee58c05 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:24.218883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 709/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:26.355930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:26.356571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:26.356571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:26.357779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 677/1386] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:26.363483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3cc-028e-481c-9205-f68ee2b3213c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:26.363973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3cc-028e-481c-9205-f68ee2b3213c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:26.364240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f3cc-028e-481c-9205-f68ee2b3213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:26.450963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e792f3cc-028e-481c-9205-f68ee2b3213c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:26.452676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e792f3cc-028e-481c-9205-f68ee2b3213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:26.563788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 710/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:26.608144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:26.609525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:26.688178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:26.772208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963b61a-879a-44c8-94f7-faf67ff2f80b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:26.892650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:26.893239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:26.893556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:26.893797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:26.895460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:26.934469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:26.989150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:26.991346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4a035-5a25-455e-b871-744fe7d7fc2c service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:27.010189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 678/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:29.232869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:29.233404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:29.233404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:29.234805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 711/1389] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:29.240717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5f5c2-1e94-4011-855a-70213678525d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:29.241157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5f5c2-1e94-4011-855a-70213678525d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:29.241264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5f5c2-1e94-4011-855a-70213678525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:29.306520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e6b5f5c2-1e94-4011-855a-70213678525d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:29.307712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e6b5f5c2-1e94-4011-855a-70213678525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:29.407055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 679/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:29.452248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:29.453749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:29.485975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:29.606707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f415a-fd13-45e6-9eef-4064123164b6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:29.709124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:29.721140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:29.721809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:29.722095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:29.722369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:29.724208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:29.814199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:29.815815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39409fe9-7060-4f3f-bb52-c12eb3455c3e service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:29.834330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 712/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:29.887179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2539-d9c6-4630-a7c0-726ab98a5445 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:29.887533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2539-d9c6-4630-a7c0-726ab98a5445 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:29.887862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2539-d9c6-4630-a7c0-726ab98a5445 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:29.889196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 680/1392] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:29.894605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245733e-d2a2-44d2-94d9-c809fe074d59 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:29.894899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245733e-d2a2-44d2-94d9-c809fe074d59 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:29.895180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245733e-d2a2-44d2-94d9-c809fe074d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:29.969934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5245733e-d2a2-44d2-94d9-c809fe074d59 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:29.971073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5245733e-d2a2-44d2-94d9-c809fe074d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:30.034642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:30.074098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 713/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:30.130563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:30.130845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:30.131122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:30.132508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 681/1394] 10.222.0.27 () {54 vars in 966 bytes} [Wed Jul 20 15:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:30.137592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a7391-39cf-4657-9670-c27f88cef757 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:30.137957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a7391-39cf-4657-9670-c27f88cef757 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:30.138307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a7391-39cf-4657-9670-c27f88cef757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:30.215534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-217a7391-39cf-4657-9670-c27f88cef757 None None] MFA Rules not processed for user `1a2388c351814d5193403a4b0d00b0af`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:30.216768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-217a7391-39cf-4657-9670-c27f88cef757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:30.293664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 714/1395] 10.222.0.27 () {58 vars in 1063 bytes} [Wed Jul 20 15:29:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:30.300996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:30.302493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:30.316068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:30.408233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41e4d7-f1f3-4bce-a457-dc6afdd90523 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:30.491679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:30.492432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:30.492711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:30.492982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:30.494606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:30.564227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:30.606419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:30.607904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b2e79-68bf-493a-8f93-324ec130d94a service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:30.626970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 682/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:31.341364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba54d-4ab6-4789-9485-dca6c427fcff None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:31.341845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba54d-4ab6-4789-9485-dca6c427fcff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:31.341946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ba54d-4ab6-4789-9485-dca6c427fcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:31.407998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e71ba54d-4ab6-4789-9485-dca6c427fcff None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:31.409183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e71ba54d-4ab6-4789-9485-dca6c427fcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:31.418217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:31.515790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 715/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:29:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:31.558469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:31.558948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:31.559055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:31.560421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 683/1398] 10.222.0.27 () {54 vars in 943 bytes} [Wed Jul 20 15:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:31.566539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d61ff-7150-47f1-9af1-10047aa08a8a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:31.566940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d61ff-7150-47f1-9af1-10047aa08a8a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:31.567261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d61ff-7150-47f1-9af1-10047aa08a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:31.642347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b33d61ff-7150-47f1-9af1-10047aa08a8a None None] MFA Rules not processed for user `6bd7c157dada45049506a091a97079bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:31.643535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b33d61ff-7150-47f1-9af1-10047aa08a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:31.730259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:31.754323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 716/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Wed Jul 20 15:29:31 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:31.764361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:31.765907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:31.909098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5a5cb-3f58-4875-960e-aeb285da6ea3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:31.926163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba615-5e59-4fd9-bb20-bff6726413c5 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:31.926474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba615-5e59-4fd9-bb20-bff6726413c5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:31.926782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba615-5e59-4fd9-bb20-bff6726413c5 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:31.928207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 717/1400] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:29:31.933030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:31.933317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:31.933591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:31.946782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:32.000958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:32.002188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:32.017724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:32.018385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:32.018670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:32.018946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:32.021208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:32.100368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 718/1401] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:32.106493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:32.108128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc2812-6f86-449c-9fc6-eac2e686d047 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:32.116635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 684/1402] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:29:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:29:32.145003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:32.146780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:32.230028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:32.286877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a80cd-1c89-440c-b1a6-0cac1f940fbc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:32.414183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:32.414860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:32.415032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:32.415413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:32.417257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:32.484896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:32.519847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:32.521469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef18aa-1cb5-4bbe-8283-ab5f65f91173 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:32.541350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 719/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:34.655119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.655726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.655726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.657208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 685/1404] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:34.662875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971628-6bd4-44a5-8e39-da28254082b7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.663294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971628-6bd4-44a5-8e39-da28254082b7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.663450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971628-6bd4-44a5-8e39-da28254082b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.734372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f1971628-6bd4-44a5-8e39-da28254082b7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:34.735501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f1971628-6bd4-44a5-8e39-da28254082b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:34.842922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 720/1405] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:34.889835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:34.891353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:34.969175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:35.055691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7455e5-5b1c-422d-9e10-b4813f2365d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:35.185214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:35.185935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:35.186228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:35.186509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:35.188199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:35.222905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:35.320677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:35.322210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754da7aa-7f42-4bbb-be45-a035a825bf3a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:35.347832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 686/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:37.456939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:37.457499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:37.457499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:37.462834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 721/1407] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:37.465843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b50f-4270-4d0f-9e6f-bf08057e1901 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:37.466336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b50f-4270-4d0f-9e6f-bf08057e1901 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:37.466417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b50f-4270-4d0f-9e6f-bf08057e1901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:37.540124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5e80b50f-4270-4d0f-9e6f-bf08057e1901 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:37.541337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5e80b50f-4270-4d0f-9e6f-bf08057e1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:37.656738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 687/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:37.706137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:37.708525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:37.749479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:37.881392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eb63a-d709-4020-9be6-6bdd12928e6f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:38.006741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:38.013128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:38.014223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:38.014695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:38.014974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:38.016680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:38.123562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:38.125430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cee35-2cec-4943-ab5a-7821ba35016f service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:38.150775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 722/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:47.654318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:48.145748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=47 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:54.453981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb3e4-5667-4005-87be-8d6081e77191 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:54.454514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb3e4-5667-4005-87be-8d6081e77191 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:54.454514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb3e4-5667-4005-87be-8d6081e77191 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.455914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 688/1410] 10.222.0.27 () {54 vars in 918 bytes} [Wed Jul 20 15:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:54.461647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247f50e-60d7-429e-85b8-44a6cc924c18 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:54.462071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247f50e-60d7-429e-85b8-44a6cc924c18 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:54.462207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247f50e-60d7-429e-85b8-44a6cc924c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.537516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e247f50e-60d7-429e-85b8-44a6cc924c18 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:54.538717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e247f50e-60d7-429e-85b8-44a6cc924c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:54.642919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 723/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jul 20 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:54.657456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe286-a8ad-467e-bd5f-633a730dad64 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:54.657741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe286-a8ad-467e-bd5f-633a730dad64 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:54.658013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe286-a8ad-467e-bd5f-633a730dad64 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.659367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 689/1412] 10.222.0.27 () {54 vars in 918 bytes} [Wed Jul 20 15:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:54.663888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb8499-8ac8-4b3a-89ac-0466dc475985 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:54.664180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb8499-8ac8-4b3a-89ac-0466dc475985 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:54.664458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb8499-8ac8-4b3a-89ac-0466dc475985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.734127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:54.736998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-66cb8499-8ac8-4b3a-89ac-0466dc475985 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:54.738805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-66cb8499-8ac8-4b3a-89ac-0466dc475985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:54.853931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 724/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jul 20 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:57.336268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:57.336812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:57.336812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:57.338212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 690/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:57.343547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57563948-ea57-4227-bb2d-aa2a374dc413 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:57.343988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57563948-ea57-4227-bb2d-aa2a374dc413 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:57.344162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57563948-ea57-4227-bb2d-aa2a374dc413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:57.408253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-57563948-ea57-4227-bb2d-aa2a374dc413 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:57.409444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-57563948-ea57-4227-bb2d-aa2a374dc413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:57.464163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:57.506187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 725/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:57.550472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:57.552940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:57.702517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae9772-439a-4a89-877d-e5e47ef8ca98 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:57.775284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:57.815366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:57.816042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:57.816325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:57.816603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:57.818229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:57.915520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:57.917234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74cd3f-2eb8-4705-87a8-1016d3b45a83 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:57.934967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 691/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:58.826787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:58.827350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:58.827350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:58.828729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 726/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:58.833537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb5e0-fa82-4833-8364-8e5e67e68814 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:58.833929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb5e0-fa82-4833-8364-8e5e67e68814 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:58.834089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb5e0-fa82-4833-8364-8e5e67e68814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:58.907284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2f0bb5e0-fa82-4833-8364-8e5e67e68814 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:58.908504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2f0bb5e0-fa82-4833-8364-8e5e67e68814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:58.935732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:59.004834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 692/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:59.153840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:59.155363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:59.299198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b834219-a3b7-49e1-a02e-68aa6b41b07a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:59.339173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:59.410408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:59.410986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:59.411267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:59.411541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:59.413181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:59.505148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:59.506730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ec03f-7870-4ab7-aa73-ec4180df38fd service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:59.524527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 727/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:00.399787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.400324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.400414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.401762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 693/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:00.406708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362766cc-7110-4be2-8fdb-256c5825e376 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.407158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362766cc-7110-4be2-8fdb-256c5825e376 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.407285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362766cc-7110-4be2-8fdb-256c5825e376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.460143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:00.482263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-362766cc-7110-4be2-8fdb-256c5825e376 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:00.483456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-362766cc-7110-4be2-8fdb-256c5825e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:00.581947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 728/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:00.730192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:00.731615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:00.875592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883201-6810-4ed1-af1d-8aba91367cba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:00.887754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:00.987391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:00.987982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.988280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.988555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.990156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:01.076689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:01.078253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0b377-e7c5-4d42-aeaa-3338c956e3f1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:01.096098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 694/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:01.961054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:01.961601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:01.961601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:01.963024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 729/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:01.967990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433bc13-7e56-4f3f-a852-1887f58f2138 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:01.968447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433bc13-7e56-4f3f-a852-1887f58f2138 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:01.968554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433bc13-7e56-4f3f-a852-1887f58f2138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:01.986874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:02.043301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d433bc13-7e56-4f3f-a852-1887f58f2138 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:02.044624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d433bc13-7e56-4f3f-a852-1887f58f2138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:02.140053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 695/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:02.289688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:02.291104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:02.400996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:02.437856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41942a9-55d1-4af1-9cf6-82a2d2fc9ebc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:02.549787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:02.550347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:02.550614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:02.551385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:02.553028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:02.627950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:02.642155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:02.643807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2145a-167f-4b4b-b13f-271520954d12 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:02.662544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 730/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:03.521652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:03.522242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:03.522242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:03.523580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 696/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:03.528605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb2ab-5779-4b51-a0df-7526011ce08a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:03.529010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb2ab-5779-4b51-a0df-7526011ce08a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:03.529169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb2ab-5779-4b51-a0df-7526011ce08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:03.603206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1acdb2ab-5779-4b51-a0df-7526011ce08a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:03.604441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1acdb2ab-5779-4b51-a0df-7526011ce08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:03.700196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 731/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:03.850731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:03.852269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:03.943126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:04.006386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c3ee-7364-4c46-9c3c-3dfa1b9e8939 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:04.117136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:04.117777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:04.118054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:04.118320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:04.119963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:04.168379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:04.211965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:04.213564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91c41c-81f6-41d3-a442-36d159967443 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:04.231891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 697/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:05.038084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:05.038660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:05.038660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:05.040071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 732/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:05.045043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e74-9d4b-4f13-bdfe-8ac4736b89df None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:05.045473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e74-9d4b-4f13-bdfe-8ac4736b89df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:05.045605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748e74-9d4b-4f13-bdfe-8ac4736b89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:05.119564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-25748e74-9d4b-4f13-bdfe-8ac4736b89df None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:05.121596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-25748e74-9d4b-4f13-bdfe-8ac4736b89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:05.230450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 698/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:05.380180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:05.381697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:05.431781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:05.527163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e93ad7-0928-4ea3-a382-12f83a38ff84 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:05.637088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:05.637657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:05.637935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:05.638205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:05.639834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:05.646631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:05.729280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:05.730921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ba518-0a64-4e21-b037-4d47a1cbef11 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:05.749440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 733/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:06.548066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:06.548624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:06.548624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:06.550029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 699/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:06.555524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae8e0-9c57-4d88-8eeb-e128ee239827 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:06.555985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae8e0-9c57-4d88-8eeb-e128ee239827 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:06.556118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae8e0-9c57-4d88-8eeb-e128ee239827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:06.630197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bc0ae8e0-9c57-4d88-8eeb-e128ee239827 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:06.631388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bc0ae8e0-9c57-4d88-8eeb-e128ee239827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:06.698952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:06.727973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 734/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:06.877578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:06.879122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:06.897534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:07.029394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da649ed0-3e5e-4cea-b5e3-aa11ede93b68 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:07.120709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:07.149122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:07.150053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:07.150341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:07.150619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:07.152894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:07.243139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:07.244778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a3d48-9468-4c9d-b047-7244ad0d6466 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:07.263226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 700/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:08.068266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.068853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.068959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.070311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 735/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:08.075172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9117-78bf-48c9-a910-3f10a3704aa5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.075565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9117-78bf-48c9-a910-3f10a3704aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.075733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9117-78bf-48c9-a910-3f10a3704aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.156718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a28a9117-78bf-48c9-a910-3f10a3704aa5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:08.157973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a28a9117-78bf-48c9-a910-3f10a3704aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.200073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:08.255469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 701/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:08.411981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:08.413496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.558897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9687ec-99d3-4cdf-aaea-8a77196c985a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:08.612255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:08.671100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:08.671676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:08.672010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:08.672287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:08.673939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:08.764022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:08.766111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8692a2-3839-45f7-a990-1457c9d8c264 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:08.784668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 736/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:09.595269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:09.595845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:09.595845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:09.597238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 702/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:09.602784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8716f-71dc-40ec-8d8b-ad97ddf08790 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:09.603167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8716f-71dc-40ec-8d8b-ad97ddf08790 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:09.603335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8716f-71dc-40ec-8d8b-ad97ddf08790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:09.679109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f4c8716f-71dc-40ec-8d8b-ad97ddf08790 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:09.680258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f4c8716f-71dc-40ec-8d8b-ad97ddf08790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:09.682318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:09.777746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 737/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:09.924919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:09.926951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:10.071218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02238961-fd4c-45aa-a616-c56a9114cffc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:10.098411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:10.186011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:10.186619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:10.186898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:10.187167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:10.188850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:10.277596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:10.279194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977da4f-125b-47af-9efa-75120c6a35ea service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:10.297696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 703/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:12.266746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.267288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.267420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.268762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 738/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:12.274263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f06e1-b264-4e13-8a8a-58152be725ba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.274765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f06e1-b264-4e13-8a8a-58152be725ba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.275074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f06e1-b264-4e13-8a8a-58152be725ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.314730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:12.342093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc8f06e1-b264-4e13-8a8a-58152be725ba None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:12.343297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc8f06e1-b264-4e13-8a8a-58152be725ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.439036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 704/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:12.477695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:12.479234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.584140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5491c0-1eaa-48a4-846a-51c7066848f9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:12.615656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:12.657302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:12.657952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:12.658236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:12.658536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:12.660257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:12.767235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:12.768706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e0c78-9fa0-4431-903c-5f75583d7316 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:12.786963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 739/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:15.060597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:15.061161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:15.061161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:15.062572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 705/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:15.067727 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4655e-1e8a-48e4-aacb-549deb424cb9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:15.068148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4655e-1e8a-48e4-aacb-549deb424cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:15.068315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4655e-1e8a-48e4-aacb-549deb424cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:15.133700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-aec4655e-1e8a-48e4-aacb-549deb424cb9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:15.134887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aec4655e-1e8a-48e4-aacb-549deb424cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:15.148127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:15.243363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 740/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:15.284596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:15.286715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:15.413595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a005a-ef94-4b32-ae39-49c93e5e476f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:15.426718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:15.488935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:15.489587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:15.489870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:15.490143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:15.491807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:15.600534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:15.601952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380af3-da3a-4443-b94c-e917add39c25 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:15.619608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 706/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:17.966879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:17.967444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:17.967444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:17.968853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 741/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:17.974590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bcf5-e3e7-417a-8144-fd65a4cbbef2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:17.975085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bcf5-e3e7-417a-8144-fd65a4cbbef2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:17.975195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bcf5-e3e7-417a-8144-fd65a4cbbef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:18.022044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:18.040378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-94d3bcf5-e3e7-417a-8144-fd65a4cbbef2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:18.041586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94d3bcf5-e3e7-417a-8144-fd65a4cbbef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:18.137330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 707/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:18.177266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:18.178783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:18.281195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807ed5-f4b6-4158-9923-c87fc982b1e6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:18.322170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:18.356506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:18.357165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:18.357454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:18.357730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:18.359351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:18.467126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:18.468586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e094a-c6de-46a2-aa16-3c76ae2ef05d service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:18.486249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 742/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:20.623330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:20.623902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:20.624283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:20.626606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 708/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:20.632411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b00c-6df9-4f6b-bc52-c8060a9b776b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:20.632806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b00c-6df9-4f6b-bc52-c8060a9b776b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:20.632964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b00c-6df9-4f6b-bc52-c8060a9b776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:20.696136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0b47b00c-6df9-4f6b-bc52-c8060a9b776b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:20.697241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0b47b00c-6df9-4f6b-bc52-c8060a9b776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:20.717489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:20.791952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 743/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:20.830556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:20.832710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:20.933903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46230170-ecaa-4de4-a88e-7967723de19b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:20.954061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:21.007295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:21.007979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:21.008265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:21.008543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:21.010162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:21.116963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:21.118387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42327a-750a-46f3-a24f-e841ff0719a7 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:21.136930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 709/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:23.288240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:23.290072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:23.290369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:23.296719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 744/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:23.298238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364517a9-ab03-4064-9b5e-794fa8262839 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:23.299100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364517a9-ab03-4064-9b5e-794fa8262839 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:23.299199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364517a9-ab03-4064-9b5e-794fa8262839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:23.362721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:23.366175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-364517a9-ab03-4064-9b5e-794fa8262839 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:23.367552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-364517a9-ab03-4064-9b5e-794fa8262839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:23.464771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 710/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:23.503590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:23.505162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:23.608232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a03b0-fac8-413d-8ef3-28d335c5e16c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:23.657313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:23.681830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:23.682478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:23.682761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:23.683037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:23.684712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:23.796234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:23.797667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb095598-8416-4230-b8a6-fbfd5b133b50 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:23.819785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 745/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:25.950138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334275-c758-4c41-8d61-150332f256ea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:25.950697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334275-c758-4c41-8d61-150332f256ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:25.950697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334275-c758-4c41-8d61-150332f256ea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:25.952108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 711/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:30:25.957426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854db5-4e73-48e3-94f6-ec7ee169d0c1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:25.957807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854db5-4e73-48e3-94f6-ec7ee169d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:25.957981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05854db5-4e73-48e3-94f6-ec7ee169d0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:26.021433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-05854db5-4e73-48e3-94f6-ec7ee169d0c1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:26.022619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-05854db5-4e73-48e3-94f6-ec7ee169d0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:26.052284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:26.121686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 746/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:26.160487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334275-c758-4c41-8d61-150332f256ea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:26.162012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b9334275-c758-4c41-8d61-150332f256ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:26.266353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334275-c758-4c41-8d61-150332f256ea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:26.294573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:26.338486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:26.339132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:26.339427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:26.339681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:26.341374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:26.448570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:26.449996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac9005-5a5d-45b5-9347-53546a05692a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:26.468377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 712/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:28.555234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.555838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.555838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.557174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 747/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:28.562525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555992-deba-4658-94db-33d20d8623fa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.562979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555992-deba-4658-94db-33d20d8623fa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.563077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555992-deba-4658-94db-33d20d8623fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.626959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9c555992-deba-4658-94db-33d20d8623fa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:28.628191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c555992-deba-4658-94db-33d20d8623fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:28.636472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.729323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 713/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:28.766128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:28.767677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:28.870095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0787fb-022a-48ff-bbe5-8beb3c097623 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:28.924393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:28.943420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:28.944101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:28.944387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:28.944662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:28.946290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:29.053624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:29.054984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71abd90-4915-4fe2-9b00-31157b8793d9 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:29.073034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 748/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:31.158074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:31.158765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:31.158765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:31.160096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 714/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:31.165496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a600d3-9380-4068-a3c7-3cdf839f53d4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:31.165896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a600d3-9380-4068-a3c7-3cdf839f53d4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:31.166051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a600d3-9380-4068-a3c7-3cdf839f53d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:31.232244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-30a600d3-9380-4068-a3c7-3cdf839f53d4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:31.233439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-30a600d3-9380-4068-a3c7-3cdf839f53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:31.272519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:31.330401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 749/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:31.369656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:31.371431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:31.476346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe911571-231e-435c-969a-67aa51f0f3ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:31.513721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:31.550535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:31.551189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:31.551474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:31.551791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:31.553421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:31.666475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:31.667874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ed07b-41ea-43a8-91cd-58f444972ca1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:31.686293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 715/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:33.758525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.759051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.759051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.760426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 750/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:33.765704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874bcf9-0dc8-4307-8061-282972dda6ee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:33.766141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874bcf9-0dc8-4307-8061-282972dda6ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:33.766265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874bcf9-0dc8-4307-8061-282972dda6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:33.829822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4874bcf9-0dc8-4307-8061-282972dda6ee None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:33.831530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4874bcf9-0dc8-4307-8061-282972dda6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:33.849546 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:33.926763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 716/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:30:33.964325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:33.965853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:34.071412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221e3b-80be-475a-809a-9365445c1562 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:34.141271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:34.149423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:34.150073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:34.150354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:34.150627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:34.152276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:34.260111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:34.261550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4e8a1-d135-4dd1-a8be-ccd5a883ad2f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:34.280154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 751/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:36.318678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:36.319239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:36.319239 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:36.320568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 717/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:36.326024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37a3f-dbc1-4a61-bc74-04963daba96a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:36.326459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37a3f-dbc1-4a61-bc74-04963daba96a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:36.326607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37a3f-dbc1-4a61-bc74-04963daba96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:36.391805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-dbd37a3f-dbc1-4a61-bc74-04963daba96a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:36.393006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dbd37a3f-dbc1-4a61-bc74-04963daba96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:36.444507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:36.494424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 752/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:36.532548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:36.534584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:36.640524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc932b0-6140-43b7-9f4b-7364e2853fd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:36.689384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:36.715253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:36.715956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:36.716230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:36.716505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:36.718138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:36.827628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:36.829071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9acd82-cf0a-4eef-a943-0bd5f5a53bc9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:36.847841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 718/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:30:46.492534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=22 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:46.844930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=66 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:56.720747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ee850-2c42-4703-94f1-d62c65e36602 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.721260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ee850-2c42-4703-94f1-d62c65e36602 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.721260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ee850-2c42-4703-94f1-d62c65e36602 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.722645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 753/1471] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:30:56.728252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e625ca1-894e-49b3-9d7d-f767c6a8ed70 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.728687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e625ca1-894e-49b3-9d7d-f767c6a8ed70 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.728796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e625ca1-894e-49b3-9d7d-f767c6a8ed70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.796973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e625ca1-894e-49b3-9d7d-f767c6a8ed70 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:56.798164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e625ca1-894e-49b3-9d7d-f767c6a8ed70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:56.899394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 719/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:56.988467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.988745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.989205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:56.990596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 754/1473] 10.222.0.27 () {54 vars in 966 bytes} [Wed Jul 20 15:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:56.996168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea721e-7499-40aa-9816-27cb1047aa3a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:56.996520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea721e-7499-40aa-9816-27cb1047aa3a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:56.996801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea721e-7499-40aa-9816-27cb1047aa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.074984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-04ea721e-7499-40aa-9816-27cb1047aa3a None None] MFA Rules not processed for user `1a2388c351814d5193403a4b0d00b0af`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:57.076283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-04ea721e-7499-40aa-9816-27cb1047aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:57.100907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:57.159721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 720/1474] 10.222.0.27 () {58 vars in 1063 bytes} [Wed Jul 20 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:57.166796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:57.168383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:57.279330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e170c1-2cbd-4e5e-a939-9468c9c3029e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:57.352649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:57.353300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:57.353585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:57.353860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:57.355490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:57.413537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:57.472983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:57.477133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0c538-a2e5-4a05-8a25-a201c9f07f76 service glance] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:57.498243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 755/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:59.639250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.639824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.639935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.641293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 721/1476] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:59.647107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea03e9-60af-49d4-a734-2e57984b5622 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:59.647513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea03e9-60af-49d4-a734-2e57984b5622 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:59.647664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea03e9-60af-49d4-a734-2e57984b5622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:59.711194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-29ea03e9-60af-49d4-a734-2e57984b5622 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:59.712337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-29ea03e9-60af-49d4-a734-2e57984b5622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:59.827878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 756/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:59.851599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:59.853154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:59.983120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef509-182f-4f19-a714-fb97f610d3b1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:00.033051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:00.078051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:00.078713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:00.078995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:00.079269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:00.080079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:00.081849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:00.081849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:31:00.081849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:31:00.083019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67bcfd-2320-424d-b50a-fd65b3e73744 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:00.101500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 722/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jul 20 15:30:59 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:02.149402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.149943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.149943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.151366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 757/1479] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:02.156521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8954-e8ed-4b4c-a2b2-c482fee9c156 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.156917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8954-e8ed-4b4c-a2b2-c482fee9c156 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.157081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8954-e8ed-4b4c-a2b2-c482fee9c156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.221706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a73c8954-e8ed-4b4c-a2b2-c482fee9c156 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:02.222911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a73c8954-e8ed-4b4c-a2b2-c482fee9c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.318446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 723/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:02.361859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:02.363731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.373935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.508592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd722a9-3327-482a-b246-11aac345d0e3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:02.586190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:02.618702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:02.619353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:02.619637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:02.619960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:02.621831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:02.709566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:02.711079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592354c2-cd9c-4b44-a8ad-786837e1c7e8 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:02.729140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 758/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:04.928577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:04.929158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:04.929158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:04.930566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 724/1482] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:04.935882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886125b-c57a-4179-a39f-ae536af9d8fa None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:04.936308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886125b-c57a-4179-a39f-ae536af9d8fa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:04.936481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886125b-c57a-4179-a39f-ae536af9d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:05.000474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8886125b-c57a-4179-a39f-ae536af9d8fa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:05.001698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8886125b-c57a-4179-a39f-ae536af9d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:05.047205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:05.097797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 759/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:05.134627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:05.136212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:05.161641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:05.240492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2bfeb-660f-45e4-9de9-c6b1262b5113 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:05.313921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:05.314682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:05.314971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:05.315246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:05.316934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:05.390386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:05.426458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:05.427922 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c61f9c-fe67-4712-935f-6d61315c2b05 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:05.447116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 725/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:07.542955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:07.543499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:07.543499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:07.544938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 760/1485] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:07.550483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc742-4bdd-4c47-91c5-ef4f460bbfb1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:07.550974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc742-4bdd-4c47-91c5-ef4f460bbfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:07.551083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc742-4bdd-4c47-91c5-ef4f460bbfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:07.616691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87afc742-4bdd-4c47-91c5-ef4f460bbfb1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:07.617900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87afc742-4bdd-4c47-91c5-ef4f460bbfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:07.713662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 726/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:07.752385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:07.753903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:07.976484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94a568-8835-45b2-be47-0a65a24ae41a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:08.046017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:08.048133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:08.048791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:08.049258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:08.049685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:08.051505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:08.161731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:08.163239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c674e11-d931-4c07-808f-e9a0476f9145 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:08.181884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 761/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:31:10.297357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:10.297891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:10.297891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:10.299232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 727/1488] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:10.304466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d15ec3-2f6d-4445-8874-fe65bedba9c3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:10.304887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d15ec3-2f6d-4445-8874-fe65bedba9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:10.305002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d15ec3-2f6d-4445-8874-fe65bedba9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:10.367542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c8d15ec3-2f6d-4445-8874-fe65bedba9c3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:10.368743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c8d15ec3-2f6d-4445-8874-fe65bedba9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:10.426760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:10.470944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 762/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:10.512075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:10.513588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:10.550171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:10.617170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee998f6-d6db-439f-88b8-b39d27f4872d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:10.691043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:10.691668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:10.691978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:10.692252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:10.693892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:10.772560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:10.802413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:10.803803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585aee2-ae4c-4e68-9f44-3377b17dc6af service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:10.822020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 728/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:13.030180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.030744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.030744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.032194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 763/1491] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jul 20 15:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:13.037289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5abe9-e5b6-400a-95b4-ee08b15afb82 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.037739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5abe9-e5b6-400a-95b4-ee08b15afb82 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.037842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5abe9-e5b6-400a-95b4-ee08b15afb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.102250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2db5abe9-e5b6-400a-95b4-ee08b15afb82 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:13.103448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2db5abe9-e5b6-400a-95b4-ee08b15afb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.199807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 729/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jul 20 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:13.239804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:13.241318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.344735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10404e7f-b874-449c-958a-027f8a50c438 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:13.422766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:13.424111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:13.424396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:13.424674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:13.426572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:13.435998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:13.534046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:13.535470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8630013-f55f-4e77-8964-9eae449ffffa service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:13.553687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 764/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:23.197959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=82 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:23.551422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=50 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:39.774082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.774695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.774695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.848349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:39.849532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.944308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 730/1494] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:39.952869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.953346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.953461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.026492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:40.027615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.121309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 765/1495] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:40.126410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.127604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.138814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b05e1-ee07-4877-92d0-aaef6ed4eb36 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.207704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:40.241867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.242527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.242809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.243078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.243806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:40.244113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.245398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:40.245398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:32:40.245398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:32:40.246266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.257900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 731/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:40.266087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.274981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514adc51-ea2f-4aa4-ab3a-eb2c1ab85f4e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.334026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:40.366842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.367486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.367792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.368060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.368744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.370704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:40.370704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:32:40.370704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:32:40.371923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.428345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 766/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:40 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:40.430160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.431668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.468288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55727c-f2c6-405b-9536-6236ca2259f7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.560129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.560796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.561079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.561347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.562064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.563067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:40.563067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:32:40.563067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:32:40.564293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.603397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 732/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:40 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:40.607650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.609730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.670792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f802-2893-4f70-9eb6-bb5fa1a91e1b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.752608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:40.763602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.764291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.764567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.764868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.765575 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.766583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:40.766583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:32:40.766583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:32:40.767447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.779507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 767/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:40.784265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.785784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.797739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac1d25-2e7f-429a-87b6-2b2d32f5119e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.822569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:40.893457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.894055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.894333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.894603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.895323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.896362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:40.896362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:32:40.896362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:32:40.897236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bec1a-a3a0-4498-8b90-cb5085f946d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.909771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 733/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:40.914002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.915593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.947851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f44006-1c5a-4ce6-9d56-a4496d5b31d9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.039273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.040182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.040461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.040733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] PATH_INFO: `/v3/projects/f404c4434b94417b953996939b97b395/users/b2fcb0dd115c4d478171ac5662334685/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.062945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f404c4434b94417b953996939b97b395, user_id=b2fcb0dd115c4d478171ac5662334685, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.064005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:41.064005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:32:41.064005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:32:41.065924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ad57e-52fc-4527-a868-8c06b53d3c76 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.089907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 768/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:40 2022] PUT /identity/v3/projects/f404c4434b94417b953996939b97b395/users/b2fcb0dd115c4d478171ac5662334685/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:41.102155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae05ee-bacf-4a89-97e3-a02073dff207 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.102447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae05ee-bacf-4a89-97e3-a02073dff207 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.102721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae05ee-bacf-4a89-97e3-a02073dff207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.157487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39ae05ee-bacf-4a89-97e3-a02073dff207 None None] MFA Rules not processed for user `b2fcb0dd115c4d478171ac5662334685`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:41.158618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39ae05ee-bacf-4a89-97e3-a02073dff207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.238161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 734/1502] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:41.252075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.252373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.252655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.254101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 769/1503] 10.222.0.27 () {54 vars in 964 bytes} [Wed Jul 20 15:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:41.259013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0fec-d872-43ee-89d0-f9660c2478ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.259303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0fec-d872-43ee-89d0-f9660c2478ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.259578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0fec-d872-43ee-89d0-f9660c2478ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.288785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:41.332472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa5a0fec-d872-43ee-89d0-f9660c2478ad None None] MFA Rules not processed for user `51875df6e7b1444cb31e793aa8727363`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:41.333651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa5a0fec-d872-43ee-89d0-f9660c2478ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.406926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 735/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Jul 20 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:41.414647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.416133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.466353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:41.520792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384b8ff-c712-4d84-907e-d8590bdb8a9e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.591135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.591794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.592077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.592348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.594013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.695674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.697183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.700641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:41.715376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 770/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:41.761917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6a671-00d4-4609-be94-783162ed4993 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.762206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6a671-00d4-4609-be94-783162ed4993 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.762481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6a671-00d4-4609-be94-783162ed4993 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.763783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 736/1506] 10.222.0.27 () {52 vars in 829 bytes} [Wed Jul 20 15:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:41.767305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.768580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.865286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a430b-5ac8-4ef3-a32c-2e54521a30b7 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.957320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.958109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.958388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.958659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] PATH_INFO: `/v3/users/b2fcb0dd115c4d478171ac5662334685` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.970570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2fcb0dd115c4d478171ac5662334685)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.971591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:41.971591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:32:41.971591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:32:41.972821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.982258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:42.035869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2fcb0dd115c4d478171ac5662334685'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.055173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2fcb0dd115c4d478171ac5662334685'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.055814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2fcb0dd115c4d478171ac5662334685'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.056168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2fcb0dd115c4d478171ac5662334685'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.067990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2fcb0dd115c4d478171ac5662334685'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.082921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 771/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:41 2022] DELETE /identity/v3/users/b2fcb0dd115c4d478171ac5662334685 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:42.087332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.087838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.087938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.143528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:42.144776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.223478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:42.237321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 737/1508] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:42.247078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.248326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.347655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27d19-d0c5-42f5-b67e-1d83dcbe5444 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.401337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:42.424161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.424832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.425117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.425393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.427045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.544947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.546808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132516f-67d5-4437-b46f-933a995a85bc service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.565383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 772/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:42.916585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.917844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.937995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e080f-fadb-4c91-891d-4f62666d45d0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.034246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.035121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.035405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.035703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] PATH_INFO: `/v3/projects/f404c4434b94417b953996939b97b395` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.045553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f404c4434b94417b953996939b97b395)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.047319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:43.047319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:32:43.047319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:32:43.048518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.106599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f404c4434b94417b953996939b97b395', 'request_id': 'req-e85b1105-eb27-4580-8744-d9b0d4005e37'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.113089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f404c4434b94417b953996939b97b395', 'request_id': 'req-e85b1105-eb27-4580-8744-d9b0d4005e37'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.126031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f404c4434b94417b953996939b97b395', 'request_id': 'req-e85b1105-eb27-4580-8744-d9b0d4005e37'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.152516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] The token cache is being invalidate because project f404c4434b94417b953996939b97b395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:43.152800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.153219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e85b1105-eb27-4580-8744-d9b0d4005e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.154642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 738/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:42 2022] DELETE /identity/v3/projects/f404c4434b94417b953996939b97b395 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:52.563119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=62 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:53.150504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:24.777674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:24.778237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:24.778237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:24.779524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 773/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:24.784468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3652ca-385c-41ab-997b-4590c97202f6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:24.784874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3652ca-385c-41ab-997b-4590c97202f6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:24.785029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3652ca-385c-41ab-997b-4590c97202f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:24.864063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fb3652ca-385c-41ab-997b-4590c97202f6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:24.865264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb3652ca-385c-41ab-997b-4590c97202f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:24.968066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 739/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:25.120957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:25.123070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:25.250127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576900-71f0-4fdd-86c9-7e8d0a822470 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:25.255658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:25.256337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:25.256624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:25.256907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:25.265059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:25.266654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81122bd-ca4c-4f19-b7c3-c922c7e85b30 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:25.275986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 774/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:26.100357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad61e06-a969-472e-8983-31ac5a87c72f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.100972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad61e06-a969-472e-8983-31ac5a87c72f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.100972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad61e06-a969-472e-8983-31ac5a87c72f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:26.102312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 740/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:26.107355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58453e1-577d-47c7-9cc2-88d36e0ab576 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.107861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58453e1-577d-47c7-9cc2-88d36e0ab576 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.108027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58453e1-577d-47c7-9cc2-88d36e0ab576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:26.126043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c58453e1-577d-47c7-9cc2-88d36e0ab576 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:26.127225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c58453e1-577d-47c7-9cc2-88d36e0ab576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:26.135048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 775/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:26.286446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad61e06-a969-472e-8983-31ac5a87c72f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:26.300687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad61e06-a969-472e-8983-31ac5a87c72f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:26.306117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:26.306736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.307017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.307301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:26.315880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:26.317465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450bbd9-c446-4377-9824-1174dbb5c9d5 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:26.327301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 741/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:26.946360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae11b-52a9-47d2-b6a2-9e77fe51582a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.946644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae11b-52a9-47d2-b6a2-9e77fe51582a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.946920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae11b-52a9-47d2-b6a2-9e77fe51582a None None] PATH_INFO: `/v3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:26.948314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 776/1517] 10.222.0.17 () {54 vars in 925 bytes} [Wed Jul 20 15:34:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:26.956685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94186ca-f7a5-461e-ac09-1d7364944706 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.956975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94186ca-f7a5-461e-ac09-1d7364944706 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.957273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94186ca-f7a5-461e-ac09-1d7364944706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:27.018930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b94186ca-f7a5-461e-ac09-1d7364944706 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:27.020163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b94186ca-f7a5-461e-ac09-1d7364944706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:27.037213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:27.090924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 742/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Jul 20 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:36.121320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=56 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:36.458621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba753bf-f10e-4579-a699-b02dc9cc0cec None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.458910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba753bf-f10e-4579-a699-b02dc9cc0cec None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.459188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba753bf-f10e-4579-a699-b02dc9cc0cec None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.460616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 777/1519] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jul 20 15:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:36.465561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924801d-fc82-49be-b70a-86e14a694fd8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.466001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924801d-fc82-49be-b70a-86e14a694fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.466123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924801d-fc82-49be-b70a-86e14a694fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.484383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d924801d-fc82-49be-b70a-86e14a694fd8 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:36.485603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d924801d-fc82-49be-b70a-86e14a694fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:36.494356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 743/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jul 20 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:36.502571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba753bf-f10e-4579-a699-b02dc9cc0cec None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:36.516978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba753bf-f10e-4579-a699-b02dc9cc0cec None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:36.521866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:36.522500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.522781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.523057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.531309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:36.532920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7bf2b-eeff-4dad-81e0-d8b0ce73425d service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:36.543644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 778/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:37.575914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b50d1-0cc8-427e-a9e6-d71ce597acc3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.576473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b50d1-0cc8-427e-a9e6-d71ce597acc3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.576566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b50d1-0cc8-427e-a9e6-d71ce597acc3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.577920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 744/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:37.582848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b7450-8ce0-47f7-b7b9-a8d60fcb1316 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.583304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b7450-8ce0-47f7-b7b9-a8d60fcb1316 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.583401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b7450-8ce0-47f7-b7b9-a8d60fcb1316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.601498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-832b7450-8ce0-47f7-b7b9-a8d60fcb1316 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:37.602691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-832b7450-8ce0-47f7-b7b9-a8d60fcb1316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:37.611388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 779/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:37.763453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b50d1-0cc8-427e-a9e6-d71ce597acc3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:37.777918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b50d1-0cc8-427e-a9e6-d71ce597acc3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:37.783923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:37.784558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.784846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.785120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.793272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:37.794936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9fa21-15f9-4f4e-82aa-f069c576734e service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:37.804918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 745/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:38.615836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66fb5-e414-483c-8219-d2d415614492 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.616393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66fb5-e414-483c-8219-d2d415614492 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.616393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66fb5-e414-483c-8219-d2d415614492 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.617787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 780/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:38.623555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81418e2-7ea8-4769-be79-d1b2c78e9906 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.623996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81418e2-7ea8-4769-be79-d1b2c78e9906 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.624163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81418e2-7ea8-4769-be79-d1b2c78e9906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.643237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d81418e2-7ea8-4769-be79-d1b2c78e9906 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:38.644487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d81418e2-7ea8-4769-be79-d1b2c78e9906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:38.652419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 746/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:38.809140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a66fb5-e414-483c-8219-d2d415614492 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.823925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a66fb5-e414-483c-8219-d2d415614492 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.829272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.829902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.830183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.830460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.838538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:38.840153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6eff2-7048-434c-94e3-83d31c14205d service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.849557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 781/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:42.841405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.842520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.842520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.843935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 747/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:42.849037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d6161-a02b-4cde-9e5f-ef3b1a87f442 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.849481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d6161-a02b-4cde-9e5f-ef3b1a87f442 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.849587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d6161-a02b-4cde-9e5f-ef3b1a87f442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.867731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-882d6161-a02b-4cde-9e5f-ef3b1a87f442 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:42.868964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-882d6161-a02b-4cde-9e5f-ef3b1a87f442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.876907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 782/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:42.949496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.951219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:43.147811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcba19-066a-45d1-830b-a1364251c387 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.151281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.151953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.152240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.152518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.161943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.163352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bf23a-0a70-451f-8df7-115c85369f43 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.172765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 748/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:45.183820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce08a-4351-43ba-b926-9ca8f86296f4 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.184399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce08a-4351-43ba-b926-9ca8f86296f4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.184399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce08a-4351-43ba-b926-9ca8f86296f4 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.186285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 783/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:45.191376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69573-cf16-4c97-9f8d-aefaaa03b3c9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.191976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69573-cf16-4c97-9f8d-aefaaa03b3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.192265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69573-cf16-4c97-9f8d-aefaaa03b3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.210480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-64f69573-cf16-4c97-9f8d-aefaaa03b3c9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:45.211706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-64f69573-cf16-4c97-9f8d-aefaaa03b3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.219452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 749/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:45.255819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce08a-4351-43ba-b926-9ca8f86296f4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:45.266993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce08a-4351-43ba-b926-9ca8f86296f4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:45.270446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:45.271078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.271358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.271633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.280393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:45.281819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477bb21f-ba69-454b-a466-9c579d44302f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:45.291131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 784/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:48.884773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:48.885329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:48.885329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:48.886722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 750/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:48.891989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3616-0a72-4ee3-af86-ac23f6b77609 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:48.892428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3616-0a72-4ee3-af86-ac23f6b77609 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:48.892547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c3616-0a72-4ee3-af86-ac23f6b77609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:48.911317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-959c3616-0a72-4ee3-af86-ac23f6b77609 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:48.912530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-959c3616-0a72-4ee3-af86-ac23f6b77609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:48.920801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 785/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:49.229122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:49.230780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:49.296864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:49.310597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a27fa-021f-47ad-9e2c-1e6161e2c745 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:49.314060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:49.314703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:49.314979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:49.315247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:49.323956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:49.325363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf17ac-5a70-4570-9ee2-80f58cb6cd58 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:49.334965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 751/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:51.373567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:51.374143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:51.374143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:51.375475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 786/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:51.380579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462132c5-4dd9-4514-a068-e8a9722770bd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:51.380997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462132c5-4dd9-4514-a068-e8a9722770bd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:51.381162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462132c5-4dd9-4514-a068-e8a9722770bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:51.398837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-462132c5-4dd9-4514-a068-e8a9722770bd None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:51.400056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-462132c5-4dd9-4514-a068-e8a9722770bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:51.408301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 752/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:51.704337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:51.705995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:51.719097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e8b52-3f8d-4f08-ac1e-1dfdb29cc93e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:51.722495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:51.723119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:51.723397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:51.723668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:51.732527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:51.733986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14467295-2af5-4e5c-af46-21a071d25470 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:51.747237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 787/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:01.394099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:01.739670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=33 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:23.960712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:23.961390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:23.961390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:23.991375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:23.992662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.001671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 753/1540] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:24.014073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.015150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.015647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.041151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.042782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.059365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 788/1541] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.064879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.083858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19437d2-34b9-4ff1-a2a0-f94d16c9164d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.096844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.097560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.097845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.098119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.098867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:24.099175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.100900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.121858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 754/1542] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:24.131299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.142020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412287b-36a0-41ff-81ac-5f501db338a2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.152839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.154544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.155060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.155567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.156555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.159092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.213526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 789/1543] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:24.218875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.240196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d6783-b495-4355-89f1-698f02c2335a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.325774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.326758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.327203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.327659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.328551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.330694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.362155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 755/1544] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:24.368356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.386857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ac9a8-0f17-44d4-957f-f3b01f5e46e6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.392613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.393430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.393845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.394131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.394869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.400444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617a3d7-272f-4169-960c-a10181db1e49 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.412755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 790/1545] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:24.416862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.434369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24a383-3481-4a37-8b57-2ad9688bc865 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.442510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c369a-0683-4f4c-b49c-8506cd31f962 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.443024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c369a-0683-4f4c-b49c-8506cd31f962 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.443471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c369a-0683-4f4c-b49c-8506cd31f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.450790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.451471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.451754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.452050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.452842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.454616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.471076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-748c369a-0683-4f4c-b49c-8506cd31f962 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.472884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-748c369a-0683-4f4c-b49c-8506cd31f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.476436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 756/1546] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:24.489936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 791/1547] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:24.491311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.498429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.498963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.499623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.515048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e03a1-fb68-4534-b6b3-a1c4a22b324f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.526866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.528403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.529709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.530621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.530900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.531174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] PATH_INFO: `/v3/projects/abad8e0d4081462bad168cbb61f83a01/users/dac164d6b76a4030ac4ebce2675dc5ad/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.545875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 792/1548] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:24.550477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.561278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e42e57-f30d-4fce-afe3-355a9614fd02 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.571019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.571751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abad8e0d4081462bad168cbb61f83a01, user_id=dac164d6b76a4030ac4ebce2675dc5ad, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.572143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.572575 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.572839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:24.572839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:36:24.572839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:36:24.573350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.574241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:24.574770 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.575504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.576814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:24.576814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:36:24.576814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:36:24.578219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41106d1b-f57f-4261-8539-33b628001362 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.600557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 793/1549] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:36:24.603536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2193-9e4e-42e2-af6d-6a2dbe39a3c9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.604898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2193-9e4e-42e2-af6d-6a2dbe39a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.605904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2193-9e4e-42e2-af6d-6a2dbe39a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.622269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 757/1550] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:24 2022] PUT /identity/v3/projects/abad8e0d4081462bad168cbb61f83a01/users/dac164d6b76a4030ac4ebce2675dc5ad/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:24.622269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.640967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a86a2193-9e4e-42e2-af6d-6a2dbe39a3c9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.642469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a86a2193-9e4e-42e2-af6d-6a2dbe39a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.714933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 794/1551] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.717000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca378e9-8adf-4e77-a613-d08f0069124a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.717245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca378e9-8adf-4e77-a613-d08f0069124a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.717525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca378e9-8adf-4e77-a613-d08f0069124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.729196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef59a-1cce-4794-8443-6f6f29af9969 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.735733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.736449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.736735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.737013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.737712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.738846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:24.738846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:36:24.738846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:36:24.739439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-dca378e9-8adf-4e77-a613-d08f0069124a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.740069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71532-0de1-4542-ae0f-4cfca21d77b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.740684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dca378e9-8adf-4e77-a613-d08f0069124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.749591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 795/1552] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.750845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b476-46af-407f-b234-0d669971d232 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.751129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b476-46af-407f-b234-0d669971d232 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.751406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272b476-46af-407f-b234-0d669971d232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.770310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 758/1553] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:24.770658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f272b476-46af-407f-b234-0d669971d232 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.771662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aa2e5-3f7f-4b91-80df-c8ad6e06de5d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.771984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aa2e5-3f7f-4b91-80df-c8ad6e06de5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.772258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aa2e5-3f7f-4b91-80df-c8ad6e06de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.772447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f272b476-46af-407f-b234-0d669971d232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.789624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-915aa2e5-3f7f-4b91-80df-c8ad6e06de5d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.790824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-915aa2e5-3f7f-4b91-80df-c8ad6e06de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.804103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:24.832027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 796/1554] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.833315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f87ed-10e0-4782-bb01-76edf3729d56 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.833607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f87ed-10e0-4782-bb01-76edf3729d56 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.833886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f87ed-10e0-4782-bb01-76edf3729d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.850957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-242f87ed-10e0-4782-bb01-76edf3729d56 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.852166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-242f87ed-10e0-4782-bb01-76edf3729d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.860206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 797/1555] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.868162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 759/1556] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.868318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e96f2-89f4-4d81-9104-20e6011a75d9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.868318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e96f2-89f4-4d81-9104-20e6011a75d9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.868318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e96f2-89f4-4d81-9104-20e6011a75d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.868569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.868569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.868569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.884703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-eb3e96f2-89f4-4d81-9104-20e6011a75d9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.885470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.885889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-eb3e96f2-89f4-4d81-9104-20e6011a75d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.886612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.894303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 798/1557] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.898669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 760/1558] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.898836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.898951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.899044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.899044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.908503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:24.915586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.917439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.925673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 799/1559] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.926474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:24.936589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979c639-e067-4020-bc17-7d31f6edbfbb None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.941381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.942030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.943053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.943053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.943365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.944458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:24.944458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:36:24.944458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:36:24.945655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9bee0-2f9f-4994-83a6-4ff4e902b0b5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.964380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a100b1c-2605-4005-ad8d-6503caea7635 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:24.966548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 800/1560] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:24.967921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd2951-ba5b-493f-9e68-5daed1e1835a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.968201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd2951-ba5b-493f-9e68-5daed1e1835a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.968477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd2951-ba5b-493f-9e68-5daed1e1835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.970305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:24.971002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.971280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.971551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.980754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:24.982145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea29904-511b-4952-aade-7f58405c4492 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:24.985921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-defd2951-ba5b-493f-9e68-5daed1e1835a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:24.987102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-defd2951-ba5b-493f-9e68-5daed1e1835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:24.991497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 761/1561] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:24.992871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.993154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.993423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:24.995271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 801/1562] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:24.997337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940244-6a76-4a7f-81ef-625c17ecfda7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:24.997622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940244-6a76-4a7f-81ef-625c17ecfda7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:24.997904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940244-6a76-4a7f-81ef-625c17ecfda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.015702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:25.016181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-12940244-6a76-4a7f-81ef-625c17ecfda7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:25.016925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:25.017342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-12940244-6a76-4a7f-81ef-625c17ecfda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:25.025402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 762/1563] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:25.029556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 802/1564] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:25.030137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.030252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73a0a-c047-4099-899b-843f1067d413 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.030252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73a0a-c047-4099-899b-843f1067d413 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.030252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e73a0a-c047-4099-899b-843f1067d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.039657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694882-8e3e-4fe6-a488-853c126bcbfc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.044312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.044977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.045263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.045535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.046260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.046563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.047835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-97e73a0a-c047-4099-899b-843f1067d413 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:25.048180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.049032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-97e73a0a-c047-4099-899b-843f1067d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:25.063336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 803/1565] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:25.063497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 763/1566] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.063602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.063602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.063789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.063888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.080875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d5d6-d1bc-439e-b2b2-f9e3f0d669c8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.085841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.086512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.086649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:25.086790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.087125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.087943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.088198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:25.089710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.096686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 804/1567] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:25.097502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.108396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 764/1568] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:25.108396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.111566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197b99-a9b2-48f2-a2bb-88fff1c730c6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.116996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d374df-45e6-4a3a-8e0b-aad0fe0ca3a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.117165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.117777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.118186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.118478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.119230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.119687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.121565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.124858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.125934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.126364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.126809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.127885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.128393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.130191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.135409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 805/1569] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.136492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.142442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 765/1570] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.143067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.153325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefe4d4-bc78-4685-91e8-bfafe20edcf1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.158651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530ee8-f51a-48c4-81c3-3fd5efac91cf admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.159100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.159859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.160277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.160684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.161457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.161768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.163716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.163968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.164606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.164940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.165212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.165951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.166266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.167988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.183937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 766/1571] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.186167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.191781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 806/1572] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.192737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.198334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85cbe-a178-4df9-8bd9-9f7b05e7f5c5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.203182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f0cb3-9898-4632-9397-9cad8df71229 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.204247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.205155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.205568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.205829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.206669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.208648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.208776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.209372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.209733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.210148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.210965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:25.211339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.215006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.228029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 807/1573] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:25.229081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.242910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c14c-c705-4fa4-a976-6390192fdb8e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.244865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 767/1574] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.245457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.302521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:25.313924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.314666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.314956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.315237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.316011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.317630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.329145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 808/1575] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:25.330616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af6d1f-1264-4fdf-8fed-127dc3168305 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.333349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.335436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.336138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.336419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.336691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.337393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.339542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.344367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb43a7-8cc2-4738-b700-0e060cc6cd92 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.349836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.350502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.350787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.351354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.351858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.354677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.375936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 768/1576] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.376624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.391171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 809/1577] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.392398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.455278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:25.466764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83dc35-84ea-4c55-b0e7-dd652d96e93a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.473685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.474934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.475225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.475503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.476249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.476952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff98b9-6d53-4f89-ab9b-435261e12d60 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.478377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.483922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.484622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.485031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.485380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.486208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.488308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.521404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 769/1578] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.522197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.524515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 810/1579] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.525659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.605911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8411312-e7f5-49f2-9aae-406dfd70affb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.611551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.612086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.612369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.612861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.613459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.615371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaf46b-ce50-4968-9814-87065e31de35 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.615502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.620282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.621222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.621538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.621879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.622570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.624551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.645419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 770/1580] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.646227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.648984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 811/1581] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.650274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.664176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0302257-7d29-40fa-8aa1-f02f7aedcfd4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.685356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:25.719434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380092c3-96b3-4c72-987e-352250f8d3ac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.724397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.725248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.725531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.725808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] PATH_INFO: `/v3/projects/32517d18e03b448f81c0cfe9eb5cd100/users/6ddc779f749d4433ab0af61108cc37ea/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.734276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.734971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.735260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.735548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.736325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.738413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.743896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32517d18e03b448f81c0cfe9eb5cd100, user_id=6ddc779f749d4433ab0af61108cc37ea, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.747253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.763073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 812/1582] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.764135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.764521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 771/1583] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:25 2022] PUT /identity/v3/projects/32517d18e03b448f81c0cfe9eb5cd100/users/6ddc779f749d4433ab0af61108cc37ea/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:25.768646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.776957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a235071-d14e-4d11-a089-e4a221e7cdb0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.838920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:25.842071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f4d1b-9af0-416e-bb84-5f096d24475d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.848223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.849174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.849464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.849740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.850436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.850552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.851193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.851477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.851840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.852457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.852557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.854435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.875632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 772/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.876591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.888303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4848a31-b0e2-4c4f-bce5-fd8ce1cea9ac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.894399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.895476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.895476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.895704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.896358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.898264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.905288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 813/1585] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 303 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.906482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.919607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a569dd-c179-424f-af72-cd8a2378a2f1 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.927213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.928022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.928281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.928609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.929372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.931494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.932679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 773/1586] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 296 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.933573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.956279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcd894-a74b-4766-a402-21f7895a4941 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.962747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 814/1587] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/users => generated 285 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:25.963913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:25.965265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.966690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.967083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.967452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.968570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.970941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11d67d-f4b3-45d3-bcb1-0e96b8c3f99d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:25.978512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ef58-8809-4653-8edf-f2b79a72142c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:25.988478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 774/1588] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:25.988478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.988478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.988478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.991689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:25.992511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:25.992796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:25.993167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:25.994055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:25.997993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.014249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 815/1589] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.015306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.024450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] MFA Rules not processed for user `6ddc779f749d4433ab0af61108cc37ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:26.025981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:26.030830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1299-a5cf-499f-8954-44dcc95ca9e0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.035877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.036721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.037070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.037442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.038330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.040075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.057937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 816/1590] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.057937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.069953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944472-4cfb-4d0b-9b5a-384655117b87 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.074975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.075644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.075956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.076232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.076963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.078558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.093517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 817/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.094421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.114478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbc8db-b368-4405-8118-ab89b8447eb1 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.120579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.121099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 775/1592] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:26.122453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.122571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.122923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.123199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.123963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.125713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.137848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 818/1593] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.138445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.143826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c8d6a-2505-4379-893e-960277b648d7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.150880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.151950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e663acb-f8d6-4430-b549-ebb9d6d90a0d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.152109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.152413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.152783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.153685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.155810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.157673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:26.159522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.160551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.160943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.161168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.162013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.165169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.172440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 776/1594] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.173500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.186453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174e379-9248-4e20-8d63-aea87e3b3f8e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.192870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 819/1595] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.193537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.194315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.194652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.195000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.196206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.198128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.200276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.210417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3b22c-ca85-411e-ade8-c7f7d993f016 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.214718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 777/1596] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.216576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.219672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.220846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.221155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.221431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.222167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.224014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.229591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a06fe5-6660-4784-a6c9-d6580b637c0c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.235980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.237077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.237439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.237785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.238833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.239939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 820/1597] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.241324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9c382-5e05-41f0-9abe-12b81e98f64f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.242621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.256310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 778/1598] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.258371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506e41-9997-498e-a3af-5f649da7247d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.258873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506e41-9997-498e-a3af-5f649da7247d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.259222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506e41-9997-498e-a3af-5f649da7247d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.261270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 779/1599] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jul 20 15:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:26.262069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506e41-9997-498e-a3af-5f649da7247d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.264274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a306d2-a5dd-43f3-af29-7d167a850196 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.271148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.272378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.272378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.272582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.273890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.275739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.278951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506e41-9997-498e-a3af-5f649da7247d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.284953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.285778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.286067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.286466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.287199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.289246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.290494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 821/1600] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.291421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.305444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 780/1601] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:26.306096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.306363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd4af-c322-4f45-83ed-e6f0245a5eab admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.311871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.312935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.313214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.313483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] PATH_INFO: `/v3/projects/5f7933bab51b4f9d9ed727c54f34e736/users/e4fc15fb3fc74eb597cdcf513cf22b2d/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.319308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d16b3-9467-4ad2-80a3-a9ee55ff1fb4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.325514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.326754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.327035 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.327799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] PATH_INFO: `/v3/projects/af2b2a05a6984a8ab7c35ffe4ef3be1b/users/587c639f761a4180aa0e1e86fbfb4272/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.330648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7933bab51b4f9d9ed727c54f34e736, user_id=e4fc15fb3fc74eb597cdcf513cf22b2d, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.333827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.347354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2b2a05a6984a8ab7c35ffe4ef3be1b, user_id=587c639f761a4180aa0e1e86fbfb4272, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.351130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.355434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 822/1602] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/5f7933bab51b4f9d9ed727c54f34e736/users/e4fc15fb3fc74eb597cdcf513cf22b2d/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.357104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.372390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 781/1603] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/af2b2a05a6984a8ab7c35ffe4ef3be1b/users/587c639f761a4180aa0e1e86fbfb4272/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.377298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.450039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f27f01-e138-4a44-9cd3-fcd19c11490c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.465386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86079ad6-1e85-40a6-b802-7f6679465250 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.474139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.476505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.476846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.477236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] PATH_INFO: `/v3/projects/750fc2d92e4b4363b4eb3600a4d2948a/users/19296e4abd544ac8ae62c6df94b6c39a/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.489057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:26.504723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] RBAC: Authorizing `identity:create_grant(project_id=750fc2d92e4b4363b4eb3600a4d2948a, user_id=19296e4abd544ac8ae62c6df94b6c39a, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.508560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43d78e-dac2-4906-8a9f-fffefcd9943e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.522579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.523526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.523838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.524160 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] PATH_INFO: `/v3/projects/84aa404215af4be4aceb7f77ba213bb6/users/204304ab4900484b9f51a9f7e9303af1/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.545480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 782/1604] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/750fc2d92e4b4363b4eb3600a4d2948a/users/19296e4abd544ac8ae62c6df94b6c39a/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.547270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.547664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.548392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.569247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84aa404215af4be4aceb7f77ba213bb6, user_id=204304ab4900484b9f51a9f7e9303af1, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.571293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.600625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 823/1605] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/84aa404215af4be4aceb7f77ba213bb6/users/204304ab4900484b9f51a9f7e9303af1/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.605485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.630676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:26.632222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:26.714607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:26.715344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedddd7-ee6b-473d-b2e3-917d2cc5c104 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.724433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.725864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.726384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.726913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] PATH_INFO: `/v3/projects/1b36bacc49184b3bb8ebcb8ee6eab876/users/aa2438a9459d4e9495d4ca19b6bacec8/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.732933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 783/1606] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jul 20 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:26.734106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:26.759979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f8fa-5a6d-47b5-ac5f-616547ca6200 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:26.761632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b36bacc49184b3bb8ebcb8ee6eab876, user_id=aa2438a9459d4e9495d4ca19b6bacec8, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.769779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24a12b-70e7-4105-8371-041e4e3b8cdf admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.771988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:26.772976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.773436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.773721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] PATH_INFO: `/v3/projects/2c19d4c485b2436f8d02f23bc51840ed/users/6f7a01353b3340d0ae4b7b7e2e4dbeec/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.795988 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c19d4c485b2436f8d02f23bc51840ed, user_id=6f7a01353b3340d0ae4b7b7e2e4dbeec, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:26.802942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4cc51-87ba-4e80-a51f-bef2199ee749 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:26.809384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 824/1607] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/1b36bacc49184b3bb8ebcb8ee6eab876/users/aa2438a9459d4e9495d4ca19b6bacec8/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.811171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cb9-4e27-4e01-aa65-9210050f41c1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.811656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cb9-4e27-4e01-aa65-9210050f41c1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.812000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cb9-4e27-4e01-aa65-9210050f41c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.827838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 784/1608] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:26 2022] PUT /identity/v3/projects/2c19d4c485b2436f8d02f23bc51840ed/users/6f7a01353b3340d0ae4b7b7e2e4dbeec/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:26.827838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce9f9e-aac4-4ce0-9374-79c0a85154c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.827838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce9f9e-aac4-4ce0-9374-79c0a85154c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.827838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce9f9e-aac4-4ce0-9374-79c0a85154c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.837028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-26153cb9-4e27-4e01-aa65-9210050f41c1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:26.840035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-26153cb9-4e27-4e01-aa65-9210050f41c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:26.860713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ccce9f9e-aac4-4ce0-9374-79c0a85154c6 None None] MFA Rules not processed for user `587c639f761a4180aa0e1e86fbfb4272`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:26.862512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ccce9f9e-aac4-4ce0-9374-79c0a85154c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:26.913023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 825/1609] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:26.913023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.913023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.913023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.950102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] MFA Rules not processed for user `19296e4abd544ac8ae62c6df94b6c39a`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:26.951333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:26.974496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 785/1610] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:26.977372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46bf9-9a67-48cc-a376-dbe3d1d38437 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:26.978671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46bf9-9a67-48cc-a376-dbe3d1d38437 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:26.979152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46bf9-9a67-48cc-a376-dbe3d1d38437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:26.991212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:27.012677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0dc46bf9-9a67-48cc-a376-dbe3d1d38437 None None] MFA Rules not processed for user `204304ab4900484b9f51a9f7e9303af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:27.013887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0dc46bf9-9a67-48cc-a376-dbe3d1d38437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:27.039894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 826/1611] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:27.041395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 786/1612] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987209-1a63-4415-8aa5-7efc4be03492 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987209-1a63-4415-8aa5-7efc4be03492 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987209-1a63-4415-8aa5-7efc4be03492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a9987209-1a63-4415-8aa5-7efc4be03492 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:27.134421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9987209-1a63-4415-8aa5-7efc4be03492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:27.142765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 787/1613] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:27.145169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.145475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.145745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.184840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f88b-5c86-44e5-b1bd-d0d86f7c5dac None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.189275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] MFA Rules not processed for user `6f7a01353b3340d0ae4b7b7e2e4dbeec`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:27.190734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:27.196229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.197026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.197315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.197585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.237152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:27.275172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 788/1614] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:27.276087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.292237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.294873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.307129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 827/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:27.328795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.344671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf254c4-9e5e-4f5a-94b1-8582286c2350 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.348024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.348924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.349308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.349855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.362246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.363876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.378439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 789/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:27.391963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.414837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4ea17-6d7a-472e-893b-1e1d022261fe service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.433443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878fad-15e7-4d6f-9429-dbe57ae2f574 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.442771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.447049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.448242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.448924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.450464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.462583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.463229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.463695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.464182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.464541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.468746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.471073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.478904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 828/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:27.479655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.492166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b900-20c6-4b18-967a-ee94eae43293 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.501617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.502616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.502616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.502616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.503318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.509651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.527861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:27.570128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 790/1618] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/projects => generated 405 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:27.570128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.594728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 829/1619] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:27.595669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.725155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdbd77-d066-4315-ac1b-8e2b8535ece7 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.732014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.732909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.733264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.733608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.735030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.737263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.754872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c452-9ca2-4ae3-bca0-3028000a7877 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.787789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 830/1620] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/projects => generated 369 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:27.788811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.868107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.868939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.872529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.872529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.896131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.897960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.922184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 791/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:27.923331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:27.938392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03783f7-2433-4768-9557-2c7b42338da8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.943347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.944074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.944358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.944637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.945338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.945671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a530d-ac58-4285-a37b-bf687b875861 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:27.947282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37486d4b-9fcc-44fc-a057-fee934d85ced admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:27.962050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:27.962761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:27.963023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:27.963297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:27.964093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:27.966589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.010021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:28.018875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 831/1622] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:28.019122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 792/1623] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:27 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:28.019424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.021977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e006d8-f923-4e80-90c0-fc344da81845 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.022273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e006d8-f923-4e80-90c0-fc344da81845 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.022562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e006d8-f923-4e80-90c0-fc344da81845 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.036179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 832/1624] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jul 20 15:36:28 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:28.037129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e006d8-f923-4e80-90c0-fc344da81845 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.039269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef0d6f-6b54-4b7d-808d-161e3cf90e56 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.054019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.054724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.055024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.055281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.056113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.058601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc03ba-ad5e-45ea-8517-53981f98db53 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.078930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e006d8-f923-4e80-90c0-fc344da81845 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.084322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.085095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.085969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.085969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.086278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.088054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.103631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 833/1625] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:28.103631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.120260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 793/1626] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:28 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:28.121888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.122186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.122458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.129213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd32a-e835-46a1-a21a-379aba4eced4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.140047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.141897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.142396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.143154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.145691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.149656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.176085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:28.177325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:28.179062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 834/1627] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:28.180261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.199539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad4b3c-b46a-4ad8-b8fc-075a5cc52880 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.220850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.222096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.222629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.223130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] PATH_INFO: `/v3/projects/676ea5ab340d45d0be790f6de60f99ab/users/8f947a77dfdc42fdb6b9f640d6699232/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.260927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=676ea5ab340d45d0be790f6de60f99ab, user_id=8f947a77dfdc42fdb6b9f640d6699232, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.264408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.299581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 835/1628] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:28 2022] PUT /identity/v3/projects/676ea5ab340d45d0be790f6de60f99ab/users/8f947a77dfdc42fdb6b9f640d6699232/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:28.300482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.350291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:28.400579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 794/1629] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:28.401456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.435694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12067a15-26c5-4ca2-bcc1-35409890d67b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.471775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da8d31-93dd-4fe0-9d9f-0ea08ef705fe admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.487752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.488873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.489327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.489764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.490663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.492674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.521952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 836/1630] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:28.523050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.534170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.535378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.535737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.536092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] PATH_INFO: `/v3/projects/738f4be292a84b849d17af336fc54968/users/cc2ab2921af44fbd95a470658b6929f8/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.541818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec26986-a964-4679-aec7-681cab2a79be admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.554606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.555544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.556026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.556465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.557045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738f4be292a84b849d17af336fc54968, user_id=cc2ab2921af44fbd95a470658b6929f8, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.557532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.559855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.562110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.587844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 837/1631] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:28.589288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.591532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 795/1632] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:28 2022] PUT /identity/v3/projects/738f4be292a84b849d17af336fc54968/users/cc2ab2921af44fbd95a470658b6929f8/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:28.593420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.790613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2195b6-da3c-4bc3-b363-8461b11d7104 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.804904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:28.817700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b28ce-d92c-4258-9806-07e7cfdebc57 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:28.839155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.840097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.840373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.840708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] PATH_INFO: `/v3/projects/1250e97b865041559716b2cb80317d0b/users/48881e95a97c40bcab01f97ca7341319/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:28.888200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1250e97b865041559716b2cb80317d0b, user_id=48881e95a97c40bcab01f97ca7341319, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:28.890875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:28.933769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 796/1633] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:28 2022] PUT /identity/v3/projects/1250e97b865041559716b2cb80317d0b/users/48881e95a97c40bcab01f97ca7341319/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:28.936176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:28.948356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:28.949018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:28.949352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:28.949636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.071853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.075497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.084863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297560-e2e1-4c59-a75d-5abde64961d4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.091456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.092745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:29.093217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.093217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.093381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] PATH_INFO: `/v3/projects/676ea5ab340d45d0be790f6de60f99ab/users/8f947a77dfdc42fdb6b9f640d6699232/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.096731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] RBAC: Authorizing `identity:create_grant(project_id=676ea5ab340d45d0be790f6de60f99ab, user_id=8f947a77dfdc42fdb6b9f640d6699232, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.101366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.110976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 838/1634] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:29.114359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:29.118331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 797/1635] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:28 2022] PUT /identity/v3/projects/676ea5ab340d45d0be790f6de60f99ab/users/8f947a77dfdc42fdb6b9f640d6699232/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:29.119426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:29.138392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb94ec6-cbb7-4640-a6e6-6aaac0d7184d service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.229067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b225b9-4436-4ddf-babd-20c8bdcf776b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.238131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.238876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.239233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.239527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.240366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.242281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb8db2-ddfb-4715-9c24-02f8aabe57ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.243149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.245113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.247241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.247751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.248880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.252550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.258134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 798/1636] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:29.267204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be55133-6477-4d91-81d7-80d2c2641c77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.268091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be55133-6477-4d91-81d7-80d2c2641c77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.272316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be55133-6477-4d91-81d7-80d2c2641c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.272432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 839/1637] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:29.272432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:29.287643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2053432-537f-4d8d-9b2c-2585a567ebf6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.315909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9be55133-6477-4d91-81d7-80d2c2641c77 None None] MFA Rules not processed for user `8f947a77dfdc42fdb6b9f640d6699232`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:29.316243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.317291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9be55133-6477-4d91-81d7-80d2c2641c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:29.320573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.321082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.321868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] PATH_INFO: `/v3/projects/1250e97b865041559716b2cb80317d0b/users/48881e95a97c40bcab01f97ca7341319/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.330417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1250e97b865041559716b2cb80317d0b, user_id=48881e95a97c40bcab01f97ca7341319, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.343555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.380086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 840/1638] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:29 2022] PUT /identity/v3/projects/1250e97b865041559716b2cb80317d0b/users/48881e95a97c40bcab01f97ca7341319/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:29.380086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:29.539949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:29.579468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 799/1639] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:29.585063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338525f3-8a4b-43e6-b42f-ee1c40c56d1f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.585546 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338525f3-8a4b-43e6-b42f-ee1c40c56d1f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.586379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338525f3-8a4b-43e6-b42f-ee1c40c56d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.633952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16442a3c-2839-44f4-824a-7575eedb7e0c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.645311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.648113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.649045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.650672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] PATH_INFO: `/v3/projects/738f4be292a84b849d17af336fc54968/users/cc2ab2921af44fbd95a470658b6929f8/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.655949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=738f4be292a84b849d17af336fc54968, user_id=cc2ab2921af44fbd95a470658b6929f8, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:29.661676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:29.665466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-338525f3-8a4b-43e6-b42f-ee1c40c56d1f None None] MFA Rules not processed for user `48881e95a97c40bcab01f97ca7341319`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:29.666980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-338525f3-8a4b-43e6-b42f-ee1c40c56d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:29.689937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 841/1640] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:29 2022] PUT /identity/v3/projects/738f4be292a84b849d17af336fc54968/users/cc2ab2921af44fbd95a470658b6929f8/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:29.690714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:29.825824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596ad92-c525-4247-8a91-097a68df0d6e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:29.839580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:29.841462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.841763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.842035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.851016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 800/1641] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:29.854090 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:29.854391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:29.854672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:29.933738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] MFA Rules not processed for user `cc2ab2921af44fbd95a470658b6929f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:29.935569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:29.936324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:30.015095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:30.017946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:30.046391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 842/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:30.100871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 801/1643] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:30.127976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:30.143321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0713b0-e6a1-497c-b412-d48cd31fc861 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:30.154765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:30.156708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:30.157205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:30.158387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:30.335983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:30.341298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6431f-f3d3-450b-b6a1-0c0ba97291f8 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:30.380949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 843/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:30.444787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:30.463052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b97f3-59dd-46ce-8ac0-620c64abdd5f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:30.474643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:30.481790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:30.484346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:30.485302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:30.520671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:30.528926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2a4e0-96fb-4dce-a8d2-48f1903f7e2b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:30.564316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 802/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:30.921814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc58c-57cd-4445-9285-9042fb7ad8e7 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:30.922246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc58c-57cd-4445-9285-9042fb7ad8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:30.925610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cc58c-57cd-4445-9285-9042fb7ad8e7 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:30.927613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 844/1646] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jul 20 15:36:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:30.935001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb629-cb00-4eb9-a0de-fb693f6a196d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:30.935516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb629-cb00-4eb9-a0de-fb693f6a196d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:30.935942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb629-cb00-4eb9-a0de-fb693f6a196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:30.979057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e9fb629-cb00-4eb9-a0de-fb693f6a196d None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:30.987833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e9fb629-cb00-4eb9-a0de-fb693f6a196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.051426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cc58c-57cd-4445-9285-9042fb7ad8e7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:31.139927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:31.208255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 803/1647] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jul 20 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:31.218918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babf083-fe79-4afd-a560-754d34f12580 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.219202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babf083-fe79-4afd-a560-754d34f12580 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.219480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babf083-fe79-4afd-a560-754d34f12580 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.220945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 804/1648] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jul 20 15:36:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:31.241490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.241778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.242051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.300267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:31.304488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.323578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cc58c-57cd-4445-9285-9042fb7ad8e7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:31.341350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:31.343452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 805/1649] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:31.345026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.346615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.347053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] PATH_INFO: `/v3/users/19296e4abd544ac8ae62c6df94b6c39a` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.358684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19296e4abd544ac8ae62c6df94b6c39a)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:31.362439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:31.401331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:31.423701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473d1d-0426-475d-9547-5b8db303d0db None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:31.440662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:31.441343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.441624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.441892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.526944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:31.526944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d869b6-789c-4166-aa9d-ed162b03b250 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:31.546977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19296e4abd544ac8ae62c6df94b6c39a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:31.560164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:31.560882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 806/1650] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:31.563240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19296e4abd544ac8ae62c6df94b6c39a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:31.564246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19296e4abd544ac8ae62c6df94b6c39a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:31.565235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19296e4abd544ac8ae62c6df94b6c39a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:31.576715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fb4167f1-7ff7-41e5-b623-9d89f4c3df62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19296e4abd544ac8ae62c6df94b6c39a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:31.603049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 845/1651] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:31 2022] DELETE /identity/v3/users/19296e4abd544ac8ae62c6df94b6c39a => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:31.610545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.610545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.610545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.665193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.665832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.666431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:31.681193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:31.685600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.727563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] MFA Rules not processed for user `dac164d6b76a4030ac4ebce2675dc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:31.738070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.820522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 807/1652] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:31.836803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:31.839822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.970815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 846/1653] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:31.971014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff331c2-f1c2-49b8-9d4f-dc3404f90c65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:31.985259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:31.986959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:31.994478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:31.998162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:31.998700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:31.999109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:32.036643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:32.038269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:32.083645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 808/1654] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:32.287647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa7064-9e57-49a9-8400-6327cf175086 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:32.300997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:32.303959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:32.304562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:32.305286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:32.331491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:32.336333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:32.379437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 847/1655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:32.886548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:32.891093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:32.935492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fa430-52bb-4aa3-9ec4-6edb712e88c3 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:32.951551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:32.952813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:32.953035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:32.953975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:32.954352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:32.955601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] PATH_INFO: `/v3/users/204304ab4900484b9f51a9f7e9303af1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:32.964574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204304ab4900484b9f51a9f7e9303af1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:32.972343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:32.972343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jul 20 15:36:32.972343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: Jul 20 15:36:32.974085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:32.982077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be2742-0f2d-44d7-be3a-12b98e9c8904 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:32.999450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:33.000553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:33.000785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:33.001242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] PATH_INFO: `/v3/projects/750fc2d92e4b4363b4eb3600a4d2948a` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:33.014902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750fc2d92e4b4363b4eb3600a4d2948a)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:33.015693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: WARNING py.warnings [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:36:33.015693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: warnings.warn(msg) Jul 20 15:36:33.015693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: Jul 20 15:36:33.016960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:33.227861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:33.230656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750fc2d92e4b4363b4eb3600a4d2948a', 'request_id': 'req-f02f1f5f-046d-456d-af87-f04e98e0f784'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.232335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204304ab4900484b9f51a9f7e9303af1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.234247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204304ab4900484b9f51a9f7e9303af1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.257815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750fc2d92e4b4363b4eb3600a4d2948a', 'request_id': 'req-f02f1f5f-046d-456d-af87-f04e98e0f784'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.277103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750fc2d92e4b4363b4eb3600a4d2948a', 'request_id': 'req-f02f1f5f-046d-456d-af87-f04e98e0f784'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.294562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:33.306172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204304ab4900484b9f51a9f7e9303af1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.330863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] The token cache is being invalidate because project 750fc2d92e4b4363b4eb3600a4d2948a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:33.331165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.332099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204304ab4900484b9f51a9f7e9303af1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.332787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.333360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6644d2e4-b4df-4e5a-bd01-e014dd795315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204304ab4900484b9f51a9f7e9303af1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.338629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 848/1656] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:32 2022] DELETE /identity/v3/projects/750fc2d92e4b4363b4eb3600a4d2948a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:33.341068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:33.342473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:33.366060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 809/1657] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:32 2022] DELETE /identity/v3/users/204304ab4900484b9f51a9f7e9303af1 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:33.374789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:33.376730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:33.377156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:33.440977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:33.442281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:33.562507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f1f5f-046d-456d-af87-f04e98e0f784 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:33.632609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 810/1658] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:33.697046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:33.698466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:33.762429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:33.763332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:33.763613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:33.763917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] PATH_INFO: `/v3/users/8f947a77dfdc42fdb6b9f640d6699232` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:33.771538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f947a77dfdc42fdb6b9f640d6699232)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:33.781936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:33.894673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32cd4e-ac25-4ed7-b222-d08d42488e63 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:33.909100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:33.912340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:33.912618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:33.912909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:33.917214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f947a77dfdc42fdb6b9f640d6699232'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.949859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:33.958254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:33.967388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f947a77dfdc42fdb6b9f640d6699232'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.968378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f947a77dfdc42fdb6b9f640d6699232'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.969841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f947a77dfdc42fdb6b9f640d6699232'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.985646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f947a77dfdc42fdb6b9f640d6699232'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:33.989030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 811/1659] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:34.036087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 849/1660] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:33 2022] DELETE /identity/v3/users/8f947a77dfdc42fdb6b9f640d6699232 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:34.815361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:34.816928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:34.833592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae23f3-8dec-47d2-90bd-f4ab24668fc6 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:34.904836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:34.906931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:34.913793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:34.940660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:34.943027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:34.944569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:34.944861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] PATH_INFO: `/v3/users/6f7a01353b3340d0ae4b7b7e2e4dbeec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:34.948381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7a01353b3340d0ae4b7b7e2e4dbeec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:34.954325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.033804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:35.078293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7a01353b3340d0ae4b7b7e2e4dbeec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.080009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7a01353b3340d0ae4b7b7e2e4dbeec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.086844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6d66-04f7-433d-9071-8cf6f2b07b90 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.098420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7a01353b3340d0ae4b7b7e2e4dbeec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.102740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.103840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.106752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.107154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7a01353b3340d0ae4b7b7e2e4dbeec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.107337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] PATH_INFO: `/v3/projects/676ea5ab340d45d0be790f6de60f99ab` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.107923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9632d63a-55d0-432c-83c1-a735c3a8cec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7a01353b3340d0ae4b7b7e2e4dbeec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.113006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=676ea5ab340d45d0be790f6de60f99ab)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.115934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.131087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 812/1661] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:34 2022] DELETE /identity/v3/users/6f7a01353b3340d0ae4b7b7e2e4dbeec => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:35.132758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.133928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.134426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.159179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:35.160415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.224691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676ea5ab340d45d0be790f6de60f99ab', 'request_id': 'req-4cd0ec74-b332-4d53-8884-1877a9d81dae'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.233977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676ea5ab340d45d0be790f6de60f99ab', 'request_id': 'req-4cd0ec74-b332-4d53-8884-1877a9d81dae'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.250308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676ea5ab340d45d0be790f6de60f99ab', 'request_id': 'req-4cd0ec74-b332-4d53-8884-1877a9d81dae'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.274416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] The token cache is being invalidate because project 676ea5ab340d45d0be790f6de60f99ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:35.274791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.276107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4cd0ec74-b332-4d53-8884-1877a9d81dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:35.280026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 850/1662] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:34 2022] DELETE /identity/v3/projects/676ea5ab340d45d0be790f6de60f99ab => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:36:35.282343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 813/1663] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:35.284654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f651a-9b9a-4f05-bfe0-d55ed7cbc71c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.285385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f651a-9b9a-4f05-bfe0-d55ed7cbc71c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.285985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f651a-9b9a-4f05-bfe0-d55ed7cbc71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.341439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2d4f651a-9b9a-4f05-bfe0-d55ed7cbc71c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:35.342798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2d4f651a-9b9a-4f05-bfe0-d55ed7cbc71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.365385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.367098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.481784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 851/1664] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:35.491327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15360-af78-4142-b491-67af526d0e81 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.491628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15360-af78-4142-b491-67af526d0e81 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.491950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15360-af78-4142-b491-67af526d0e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.504029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b0ee4-0fab-43e1-8525-2355ac929a59 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.534782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-34a15360-af78-4142-b491-67af526d0e81 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:35.538147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-34a15360-af78-4142-b491-67af526d0e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.557176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 852/1665] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:35.559475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15360-af78-4142-b491-67af526d0e81 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.584173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15360-af78-4142-b491-67af526d0e81 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.598498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.599440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.599895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.600357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.601935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:35.602443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.605636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.638631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 853/1666] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:35.640367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.660106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.664366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.664804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.666947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.668791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821467-1029-4567-b6d8-bf223dbd1048 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.670155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.683042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.684740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.686949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.687433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.703029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.707535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.710321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.712066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.729074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 814/1667] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:35.744349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:35.784127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 854/1668] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:35 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:35.784265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.790929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:35.797178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d964e21-8936-40c9-8f4c-2857aa486307 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.826124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.829569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:35.925650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.926193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.926478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.926754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.927489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.930218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.957465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23f1a8-705f-4419-b490-5f1bd59e056a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:35.963804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 815/1669] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:35 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:36:35.967844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:35.971055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:35.971982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:35.972264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:35.976054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] PATH_INFO: `/v3/projects/84aa404215af4be4aceb7f77ba213bb6` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:35.977368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84aa404215af4be4aceb7f77ba213bb6)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:35.979293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:35.997732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8abac8-dbc5-432c-92e9-918f71e50b9c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.007693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.011453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.013331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.014056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.016085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.018869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:36.038964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 816/1670] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:36.040673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:36.051561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84aa404215af4be4aceb7f77ba213bb6', 'request_id': 'req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.056467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743554-0f2a-4c56-8973-3251eb6fc844 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.062352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84aa404215af4be4aceb7f77ba213bb6', 'request_id': 'req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.073209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84aa404215af4be4aceb7f77ba213bb6', 'request_id': 'req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.120492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] The token cache is being invalidate because project 84aa404215af4be4aceb7f77ba213bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:36.121383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.125218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.130381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 855/1671] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:35 2022] DELETE /identity/v3/projects/84aa404215af4be4aceb7f77ba213bb6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:36:36.131380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:36.145026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:36.154226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.155283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.155741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.156173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.156985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.159459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:36.182676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 817/1672] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:36.185195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:36.187535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:36.221103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572140-4d00-4171-b009-f8c30a9a3ef0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.229932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.231385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.231673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.231975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] PATH_INFO: `/v3/projects/459425832a8f4d788d17297c2bc93df4/users/bc14bba8d40247cd80fd23cfb1c9f073/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.257989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459425832a8f4d788d17297c2bc93df4, user_id=bc14bba8d40247cd80fd23cfb1c9f073, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.262877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a20ad-ec5d-466e-8a20-4d46be8ba865 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:36.281886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f850a9c-4441-4f4e-b9f8-1b110eb8227c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.315039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 818/1673] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:36 2022] PUT /identity/v3/projects/459425832a8f4d788d17297c2bc93df4/users/bc14bba8d40247cd80fd23cfb1c9f073/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:36.326352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.326641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.326923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.353323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:36.408909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] MFA Rules not processed for user `bc14bba8d40247cd80fd23cfb1c9f073`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:36.412628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:36.442240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.443256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.445750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.446064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] PATH_INFO: `/v3/users/cc2ab2921af44fbd95a470658b6929f8` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.452338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2ab2921af44fbd95a470658b6929f8)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.458509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:36.498257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:36.546735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 819/1674] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:36.549276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:36.577583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2ab2921af44fbd95a470658b6929f8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.578866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c09b6-a841-43b7-9931-570d8549a17a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.592332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.593137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.593427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.593844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.594566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.599479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:36.604164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2ab2921af44fbd95a470658b6929f8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.605473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2ab2921af44fbd95a470658b6929f8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.607217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2ab2921af44fbd95a470658b6929f8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.621345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2ab2921af44fbd95a470658b6929f8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:36.657702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 856/1675] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:36 2022] DELETE /identity/v3/users/cc2ab2921af44fbd95a470658b6929f8 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:36.667981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 820/1676] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:36 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:36.670049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:36.671982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:36.808869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cba02-1c4b-4a3d-ab58-4743fa11a2f7 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:36.910475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:36.925342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:36.926473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:36.927378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:36.928343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:36.929297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:36.933404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.002199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 857/1677] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:36 2022] POST /identity/v3/users => generated 288 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:36:37.005433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.044394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bba702-9414-49cb-99f9-15df91f3ccfd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.055717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.057479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.057967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.058492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.060258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.062179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ba8a2-fd80-44b7-bb97-50f093e382d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.079126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 821/1678] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:37.083101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.101050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73280-5b38-4979-a240-9d254f76035f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.121483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.121880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.122077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.122459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.122997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.123371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.123659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] PATH_INFO: `/v3/projects/3c0c8867869e4009b17356b4af625cfb/users/7b15c0011ff245d2baac4ea8b0d4a362/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.143780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0c8867869e4009b17356b4af625cfb, user_id=7b15c0011ff245d2baac4ea8b0d4a362, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.146548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.165517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 858/1679] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:37 2022] PUT /identity/v3/projects/3c0c8867869e4009b17356b4af625cfb/users/7b15c0011ff245d2baac4ea8b0d4a362/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:37.169459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.174179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] MFA Rules not processed for user `dac164d6b76a4030ac4ebce2675dc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:37.177733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:37.308979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4eebb-5f3e-4ede-9c80-eadf9f548f0a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.309127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 822/1680] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:37.309127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.309485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:37.319021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.319829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.320032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.320314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.323093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.327574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.345287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db339b1-48fe-464f-adf2-03e8939a9898 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.363878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.364777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.365058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.365335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] PATH_INFO: `/v3/projects/2c19d4c485b2436f8d02f23bc51840ed` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.366298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 859/1681] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:37.367076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.373669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:37.374816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c19d4c485b2436f8d02f23bc51840ed)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.376800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.402370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a627-082f-44fa-81b1-c5a69c280c36 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.410354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.411537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.412112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.412632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] PATH_INFO: `/v3/projects/738f4be292a84b849d17af336fc54968` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.420220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738f4be292a84b849d17af336fc54968)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.422615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.511367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c19d4c485b2436f8d02f23bc51840ed', 'request_id': 'req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.524465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c19d4c485b2436f8d02f23bc51840ed', 'request_id': 'req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.543138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c19d4c485b2436f8d02f23bc51840ed', 'request_id': 'req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.568613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] The token cache is being invalidate because project 2c19d4c485b2436f8d02f23bc51840ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:37.568857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.569889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.580205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738f4be292a84b849d17af336fc54968', 'request_id': 'req-854e4c05-e098-45cc-bec7-4fb1e4a77b45'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.594465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738f4be292a84b849d17af336fc54968', 'request_id': 'req-854e4c05-e098-45cc-bec7-4fb1e4a77b45'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.600995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 823/1682] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:37 2022] DELETE /identity/v3/projects/2c19d4c485b2436f8d02f23bc51840ed => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:37.604944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.615645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738f4be292a84b849d17af336fc54968', 'request_id': 'req-854e4c05-e098-45cc-bec7-4fb1e4a77b45'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.628829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:37.633594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] The token cache is being invalidate because project 738f4be292a84b849d17af336fc54968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:37.634066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.636784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:37.640314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 860/1683] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:37 2022] DELETE /identity/v3/projects/738f4be292a84b849d17af336fc54968 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:37.641729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.645229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:37.746940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:37.748350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:37.795230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e4c05-e098-45cc-bec7-4fb1e4a77b45 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.808370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.809931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.810215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.810746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] PATH_INFO: `/v3/users/48881e95a97c40bcab01f97ca7341319` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.812904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48881e95a97c40bcab01f97ca7341319)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.818749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.847869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37a3b2-6d65-4459-a94a-a9bb8b0d866f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.874897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.875925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.876201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.876544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] PATH_INFO: `/v3/projects/3c0c8867869e4009b17356b4af625cfb/users/7b15c0011ff245d2baac4ea8b0d4a362/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.880192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0c8867869e4009b17356b4af625cfb, user_id=7b15c0011ff245d2baac4ea8b0d4a362, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.884511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313e697-1fa1-411c-8843-75f1959012fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.907956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 824/1684] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:37 2022] PUT /identity/v3/projects/3c0c8867869e4009b17356b4af625cfb/users/7b15c0011ff245d2baac4ea8b0d4a362/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:37.907956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983ff43-2690-4417-bd5e-37a9c28fc313 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.907956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983ff43-2690-4417-bd5e-37a9c28fc313 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.907956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983ff43-2690-4417-bd5e-37a9c28fc313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.925422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9983ff43-2690-4417-bd5e-37a9c28fc313 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:37.926802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9983ff43-2690-4417-bd5e-37a9c28fc313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.020439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48881e95a97c40bcab01f97ca7341319'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:38.049559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48881e95a97c40bcab01f97ca7341319'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:38.050115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48881e95a97c40bcab01f97ca7341319'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:38.050760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48881e95a97c40bcab01f97ca7341319'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:38.068070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-266302ee-d662-4f6f-9a8b-be22e24a39f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48881e95a97c40bcab01f97ca7341319'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:38.095021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 825/1685] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:38.095608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 861/1686] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:37 2022] DELETE /identity/v3/users/48881e95a97c40bcab01f97ca7341319 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:38.095690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.095690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.095690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.097490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.097929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.098209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.142410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:38.143791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.174168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] MFA Rules not processed for user `7b15c0011ff245d2baac4ea8b0d4a362`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:38.175975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.277492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 862/1687] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:38.279126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.295874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f839-b494-4133-93c3-666bb42ecba3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.304086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.304822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.305110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.305628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.306370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:38.306677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.308444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.339677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 826/1688] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:38.343020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 863/1689] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:36:38.350299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.353711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.355424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.390442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e80ce-89d8-466f-88eb-34c21f7dc503 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.430553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.431260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.431536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.437003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.440296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.443051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51896b9f-e916-4564-84da-b44d5a44994e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.473361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:38.482895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74ce46-3047-408c-8f4c-d1377eff2e17 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.494863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.496311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.496952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.497449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.517720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 827/1690] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:38 2022] POST /identity/v3/projects => generated 357 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:38.517720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124c69d-014e-454c-b3db-8c27a0569d81 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.517720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124c69d-014e-454c-b3db-8c27a0569d81 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.520727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124c69d-014e-454c-b3db-8c27a0569d81 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.522580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 828/1691] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jul 20 15:36:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:38.523375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124c69d-014e-454c-b3db-8c27a0569d81 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.524463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.528514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5996a9c-9179-4d1f-8851-6f54afc1a771 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.554610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 864/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:38.556533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.557025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.559242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.620062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:38.644247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:38.645590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.669867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124c69d-014e-454c-b3db-8c27a0569d81 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.680371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.681068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.681349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.681623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.682325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.688540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.737144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 829/1693] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:38 2022] POST /identity/v3/users => generated 279 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:38.738290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.757989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348c48e-1021-4a46-a284-fb729f100504 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.770872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.773086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.773086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.773086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.773086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.773767 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 865/1694] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:38.774768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.799792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.801484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:38.805118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 830/1695] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:38.808675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.819801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640230-8ef1-4f53-91dd-ccca60a6817d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.831774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.832606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.832930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d51ee-b0bc-405e-9a33-44b606db5e95 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.833044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.833459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.849589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.850106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.851667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.852529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.852841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.854738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.855954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.863401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0300639-4735-4dcf-846f-3105aafc3c03 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:38.877212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 866/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:38.905686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 831/1697] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:38.909187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:38.927790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fb27-9746-4606-b03d-4650eddd94e1 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:38.950267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:38.951340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:38.951849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:38.952241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] PATH_INFO: `/v3/projects/5a8d856bbccf432aad87a98177312018/users/5fd4bb188b174b039e23582648f21fc1/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:38.985745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8d856bbccf432aad87a98177312018, user_id=5fd4bb188b174b039e23582648f21fc1, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:38.988716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.031846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 867/1698] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:38 2022] PUT /identity/v3/projects/5a8d856bbccf432aad87a98177312018/users/5fd4bb188b174b039e23582648f21fc1/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:36:39.033825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.034125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.034405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.073730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.077122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.116131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] MFA Rules not processed for user `5fd4bb188b174b039e23582648f21fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:39.120707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.223118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692c851-b7b3-40f8-b665-5f25a1ddd93d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.238663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.242186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.242483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.242782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] PATH_INFO: `/v3/projects/1250e97b865041559716b2cb80317d0b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.246985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1250e97b865041559716b2cb80317d0b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.253215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.344160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 832/1699] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:39.348563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.370041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc6013-6b71-4bbf-a3d2-d9d2c1080441 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.381774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1250e97b865041559716b2cb80317d0b', 'request_id': 'req-35c3300a-4505-47eb-b08f-65077c8ec4a9'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.385419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:39.409049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1250e97b865041559716b2cb80317d0b', 'request_id': 'req-35c3300a-4505-47eb-b08f-65077c8ec4a9'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.442242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1250e97b865041559716b2cb80317d0b', 'request_id': 'req-35c3300a-4505-47eb-b08f-65077c8ec4a9'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.469875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] The token cache is being invalidate because project 1250e97b865041559716b2cb80317d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:39.472694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.477377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-35c3300a-4505-47eb-b08f-65077c8ec4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.488607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 868/1700] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:39 2022] DELETE /identity/v3/projects/1250e97b865041559716b2cb80317d0b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:39.490441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.490739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.491021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.542037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:39.542901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.543627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.543938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.544213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.544908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.545009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.546830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f50f91-e33b-46f8-b68b-b1634398fcb4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.588266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 869/1701] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:39.588990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.597807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.626340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa59032-bed6-45c5-8302-94f77725962f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.632476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 833/1702] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/projects => generated 357 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:36:39.634276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.634630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.634959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.670276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:39.673782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.707070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:39.802821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 834/1703] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:39.803661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.806161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.815018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.816116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.816395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.816676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] PATH_INFO: `/v3/users/bc14bba8d40247cd80fd23cfb1c9f073` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.821053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc14bba8d40247cd80fd23cfb1c9f073)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.826474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.834974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b99499-c0f0-45db-be3c-f220bec3a03a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.847735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.848448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.848729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.849004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.849808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.852396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.897100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 835/1704] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:39.898221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.910057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc14bba8d40247cd80fd23cfb1c9f073'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.920034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d9afd-997a-4c50-8920-997645f6db0b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.926639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc14bba8d40247cd80fd23cfb1c9f073'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.927773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc14bba8d40247cd80fd23cfb1c9f073'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.928683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc14bba8d40247cd80fd23cfb1c9f073'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.931596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.932424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.932815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.933197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.934584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:39.935001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.936750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.938353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc14bba8d40247cd80fd23cfb1c9f073'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:39.966303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 836/1705] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:39.967278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.975118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 870/1706] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:39 2022] DELETE /identity/v3/users/bc14bba8d40247cd80fd23cfb1c9f073 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:39.975118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.976478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.979064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.998271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e3028-07ce-4349-9e87-a67389b84fd8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.119189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.120054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.120445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.121100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.122512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.124134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:40.126346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42931d7-5acc-43bf-83d8-5967cfd2bd52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.159878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 837/1707] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:40.159878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.159878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.159878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.162850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450aa19-6e2e-484a-b201-c6142f2e31f2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.176376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.177372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.181493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.182016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.184027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.186703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4027e-6faa-4af8-87c6-4e3a3244ceac admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.199717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:40.201634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.219223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 838/1708] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:40.220072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.243568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda58d09-b449-4c15-99c6-d54101804b97 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.253887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.255149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.255437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.255787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] PATH_INFO: `/v3/projects/c20ba08560104d658730541ee0296273/users/018cca0fff814804bba0962b89053ec8/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.290092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20ba08560104d658730541ee0296273, user_id=018cca0fff814804bba0962b89053ec8, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.294258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 871/1709] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:39 2022] POST /identity/v3/projects => generated 387 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:40.297104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.297414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d789d1a-da7e-482a-89c4-ed5fc323d64a None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.297729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d789d1a-da7e-482a-89c4-ed5fc323d64a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.298018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d789d1a-da7e-482a-89c4-ed5fc323d64a None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.307421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 872/1710] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jul 20 15:36:40 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:40.308420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d789d1a-da7e-482a-89c4-ed5fc323d64a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.329710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 839/1711] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:40 2022] PUT /identity/v3/projects/c20ba08560104d658730541ee0296273/users/018cca0fff814804bba0962b89053ec8/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:40.335955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.338340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.442438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a772436-23ea-4221-8007-0a296b828d71 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.442856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d789d1a-da7e-482a-89c4-ed5fc323d64a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.448820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.449312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.449711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.450065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.450402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.450507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.451270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.453883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.454616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2729d6-3a97-41dc-a73b-69e71fcbd27a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.454749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.469142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:40.475144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.482005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.482553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 873/1712] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:40 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:40.484989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.485450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.485734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.515152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:40.516786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.538754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 840/1713] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:40.539192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.564793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66889f5-0f45-481e-ad2d-b7b615ecc555 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.580869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.581780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.582062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.582340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.583078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.585765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.617613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 841/1714] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:40.617613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.644264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316086ba-ad58-4a34-a181-90042090ae46 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.656179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.657129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.657366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.658026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.659890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.663191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 874/1715] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jul 20 15:36:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:40.663910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.664987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.686762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 842/1716] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:40.688185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.688805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838252e6-9799-49cc-8f12-b08b2e2ff810 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.699252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.701557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.703043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.703975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] PATH_INFO: `/v3/projects/c20ba08560104d658730541ee0296273/users/018cca0fff814804bba0962b89053ec8/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.710678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20ba08560104d658730541ee0296273, user_id=018cca0fff814804bba0962b89053ec8, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.714001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf0343-ce82-420f-831b-54a5b623e2de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.715252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.727018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.727748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.728121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.728470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.733446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.736230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 875/1717] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:40 2022] PUT /identity/v3/projects/c20ba08560104d658730541ee0296273/users/018cca0fff814804bba0962b89053ec8/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:40.737746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.765871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.767715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1ac48-3d8b-4bc3-8409-d67cbef33ed0 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.790914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 843/1718] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:40.796269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.800116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.800605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.866275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] MFA Rules not processed for user `018cca0fff814804bba0962b89053ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:40.869190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:40.876139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:40.895246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee68b3a-52a4-44dd-9917-8f05b832aca4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.909043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.909950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.910334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.911881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:40.915986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:40.919041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:40.952787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 876/1719] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:40.957194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:40.969459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c73146-08a7-49df-9d40-e718b669669b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:40.982857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:40.983995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:40.985126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:40.985547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] PATH_INFO: `/v3/projects/56089dea304149f4812013484a62fc32/users/09d8c5d78bb0466b805b5bc05572f69d/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:41.014091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56089dea304149f4812013484a62fc32, user_id=09d8c5d78bb0466b805b5bc05572f69d, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:41.017704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9652b6-22b3-43dc-9abf-0d39592acc10 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:41.045067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 877/1720] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:40 2022] PUT /identity/v3/projects/56089dea304149f4812013484a62fc32/users/09d8c5d78bb0466b805b5bc05572f69d/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:41.055673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:41.059991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:41.060479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:41.063157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 844/1721] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:40 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:41.075956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:41.078646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:41.139008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] MFA Rules not processed for user `09d8c5d78bb0466b805b5bc05572f69d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:41.143791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:41.232052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72233eca-3ce4-489f-9857-ec7d09713ebf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:41.323179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 878/1722] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:41.335980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:41.390619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:41.391859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:41.392368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:41.393069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:41.451199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:41.570129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abccc7a-ccf1-472c-bceb-162acc58c989 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:41.582474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:41.583468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:41.584129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:41.584521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:41.586450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:41.588340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:41.589193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:41.593553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:41.618262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 845/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:41.681599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 879/1724] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:41 2022] POST /identity/v3/projects => generated 385 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:41.688558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:41.712890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d34dbc-e259-4496-9519-53bee8f1995b service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:41.827140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:41.912214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:41.914386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:41.915088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:41.916719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:41.917443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:41.921844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:41.976235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 846/1725] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:41 2022] POST /identity/v3/users => generated 299 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:41.977375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:41.978094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:42.004378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2437f1-1acc-4f69-a10a-067a7ac03d66 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:42.016982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:42.019653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.020616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.021603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.024572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:42.030881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:42.031062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665156a0-8dbb-4384-9e98-937ba7fcb7a7 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:42.042436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:42.043123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.043471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.046655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.061058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:42.081120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 847/1726] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:42.082028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:42.094994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd5007-b4c3-41ba-8991-b623f91e0062 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:42.111385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:42.114710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.115033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.115398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] PATH_INFO: `/v3/projects/5c17597384fc402bbce04b723d4cfef8/users/40ae05d9f5924822b6a3b704b311bbc2/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.157666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c17597384fc402bbce04b723d4cfef8, user_id=40ae05d9f5924822b6a3b704b311bbc2, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:42.161638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:42.197373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 848/1727] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:42 2022] PUT /identity/v3/projects/5c17597384fc402bbce04b723d4cfef8/users/40ae05d9f5924822b6a3b704b311bbc2/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:42.198401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:42.236670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:42.238310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:42.267795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 880/1728] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:42.392447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9626-dea9-47f6-9c7e-8ec7127100d5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:42.409257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:42.409968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.410253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.410530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.411267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:42.413452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74699d23-9e51-4bc3-9dea-16a507170c5d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:42.482010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 849/1729] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:42.486048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:42.506080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3783e-8e20-455a-b1e3-227662009e86 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:42.520462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:42.522538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.528643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.529096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] PATH_INFO: `/v3/projects/5c17597384fc402bbce04b723d4cfef8/users/40ae05d9f5924822b6a3b704b311bbc2/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.534581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c17597384fc402bbce04b723d4cfef8, user_id=40ae05d9f5924822b6a3b704b311bbc2, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:42.542945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:42.583063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 881/1730] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:42 2022] PUT /identity/v3/projects/5c17597384fc402bbce04b723d4cfef8/users/40ae05d9f5924822b6a3b704b311bbc2/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:42.584899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:42.585601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:42.585898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:42.592666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:42.691248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] MFA Rules not processed for user `40ae05d9f5924822b6a3b704b311bbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:42.699370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:42.993323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 850/1731] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:43.012714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:43.195783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949bcf-87a4-48bb-89ed-ef3382ee4ea9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:43.201930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:43.203006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:43.203464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:43.203928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:43.206045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:43.207656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:43.249584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:43.251147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:43.283810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 882/1732] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:43.403131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6f36f-d702-4fe9-8852-6d03f363aa56 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:43.411804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:43.412976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:43.413409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:43.413817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] PATH_INFO: `/v3/projects/459425832a8f4d788d17297c2bc93df4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:43.415342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459425832a8f4d788d17297c2bc93df4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:43.418852 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:43.504264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:43.523898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b50d97-cac9-48f0-847b-f6f22713df3d service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:43.551978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:43.553589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:43.554101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:43.555103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] PATH_INFO: `/v3/users/09d8c5d78bb0466b805b5bc05572f69d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:43.557931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d8c5d78bb0466b805b5bc05572f69d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:43.563242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:43.605574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459425832a8f4d788d17297c2bc93df4', 'request_id': 'req-705eb4c5-0dca-4bf0-bf67-05b6031ab745'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.646052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459425832a8f4d788d17297c2bc93df4', 'request_id': 'req-705eb4c5-0dca-4bf0-bf67-05b6031ab745'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.684605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459425832a8f4d788d17297c2bc93df4', 'request_id': 'req-705eb4c5-0dca-4bf0-bf67-05b6031ab745'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.726205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] The token cache is being invalidate because project 459425832a8f4d788d17297c2bc93df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:43.726519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.727796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.728530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d8c5d78bb0466b805b5bc05572f69d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.732824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 851/1733] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:43 2022] DELETE /identity/v3/projects/459425832a8f4d788d17297c2bc93df4 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:43.734875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:43.740411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:43.756123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d8c5d78bb0466b805b5bc05572f69d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.756864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d8c5d78bb0466b805b5bc05572f69d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.757921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d8c5d78bb0466b805b5bc05572f69d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.772463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d8c5d78bb0466b805b5bc05572f69d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:43.776816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:43.781723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:43.822905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 883/1734] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:43 2022] DELETE /identity/v3/users/09d8c5d78bb0466b805b5bc05572f69d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:43.824007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:43.832361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:43.938663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705eb4c5-0dca-4bf0-bf67-05b6031ab745 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:44.036797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea0ba-97d8-470e-877c-b21754fc0f4e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:44.043390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:44.044533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.045130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.045560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] PATH_INFO: `/v3/users/7b15c0011ff245d2baac4ea8b0d4a362` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.058446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b15c0011ff245d2baac4ea8b0d4a362)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:44.069937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:44.174002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:44.175087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.175213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.179675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] PATH_INFO: `/v3/users/6ddc779f749d4433ab0af61108cc37ea` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.181610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddc779f749d4433ab0af61108cc37ea)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:44.183615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b15c0011ff245d2baac4ea8b0d4a362'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.188155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:44.239611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b15c0011ff245d2baac4ea8b0d4a362'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.241525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b15c0011ff245d2baac4ea8b0d4a362'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.243542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b15c0011ff245d2baac4ea8b0d4a362'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.266193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b5d53ce7-5069-4ffc-a705-e203474ca2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b15c0011ff245d2baac4ea8b0d4a362'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.299798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 884/1735] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:43 2022] DELETE /identity/v3/users/7b15c0011ff245d2baac4ea8b0d4a362 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:44.302097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.303484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.304211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.345827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddc779f749d4433ab0af61108cc37ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.348160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddc779f749d4433ab0af61108cc37ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.366920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:44.372771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:44.387467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddc779f749d4433ab0af61108cc37ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.406595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddc779f749d4433ab0af61108cc37ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.409283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-90eaa317-639c-42e1-a3f9-b99a759ef5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddc779f749d4433ab0af61108cc37ea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:44.442943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 852/1736] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:43 2022] DELETE /identity/v3/users/6ddc779f749d4433ab0af61108cc37ea => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:44.447055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.448624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.450399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.501417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:44.508339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:44.554250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 885/1737] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:44.578803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:44.580318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:44.625780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:44.668399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:44.725263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 853/1738] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:44.810773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610f067-0ee8-4913-b41e-f1a4c7f799ae None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:44.814076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:44.929929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:44.930617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.930905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.931513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.954333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbf287-2956-4f44-b867-7fc477a1bba7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:44.960944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:44.962440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1de871-509d-45b6-aba9-22d4d0e73819 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:44.962795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:44.963470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.963753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.964123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.984598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:44.986066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:45.008419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 886/1739] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:45.009785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d720-4408-48ed-8b64-7b3875f31b1f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.010090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d720-4408-48ed-8b64-7b3875f31b1f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.015635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d720-4408-48ed-8b64-7b3875f31b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:45.016617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 854/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:45.091119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-26b0d720-4408-48ed-8b64-7b3875f31b1f None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:45.093951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-26b0d720-4408-48ed-8b64-7b3875f31b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:45.218211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 887/1741] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:45.495509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:45.498590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:45.510847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.511915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.512449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:45.598946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] MFA Rules not processed for user `e4fc15fb3fc74eb597cdcf513cf22b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:45.601771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:45.690147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:45.777242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5e5a7-70a9-42fb-92b4-5834dd6ebd3f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:45.780870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 888/1742] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:45.781932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:45.785245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:45.790643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:45.791492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.794553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.794949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:45.815795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:45.818708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5129f0c-8320-44a8-ad49-34464e6c9abd service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:45.820097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49e4a-bdc6-4c35-93c2-4a5a2a4f5245 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:45.836164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:45.837486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.837486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.839443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] PATH_INFO: `/v3/projects/3c0c8867869e4009b17356b4af625cfb` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:45.841154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0c8867869e4009b17356b4af625cfb)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:45.846615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 855/1743] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:36:45.856794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c31f-3b7d-47c1-8b63-53cb9af0a4c1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.858102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:45.860172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c31f-3b7d-47c1-8b63-53cb9af0a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.860614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294c31f-3b7d-47c1-8b63-53cb9af0a4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:45.907350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4294c31f-3b7d-47c1-8b63-53cb9af0a4c1 None None] MFA Rules not processed for user `e4fc15fb3fc74eb597cdcf513cf22b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:45.908607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4294c31f-3b7d-47c1-8b63-53cb9af0a4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:45.962717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 856/1744] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:45.964338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12f48-62e8-4055-9ad0-88758566e8b9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:45.964683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12f48-62e8-4055-9ad0-88758566e8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:45.965804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a12f48-62e8-4055-9ad0-88758566e8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.046819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0c8867869e4009b17356b4af625cfb', 'request_id': 'req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:46.068432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0c8867869e4009b17356b4af625cfb', 'request_id': 'req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:46.092693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f2a12f48-62e8-4055-9ad0-88758566e8b9 None None] MFA Rules not processed for user `aa2438a9459d4e9495d4ca19b6bacec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:46.103850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0c8867869e4009b17356b4af625cfb', 'request_id': 'req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:46.116183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f2a12f48-62e8-4055-9ad0-88758566e8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.167072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] The token cache is being invalidate because project 3c0c8867869e4009b17356b4af625cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:46.169498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:46.174022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:46.181845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 889/1745] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:45 2022] DELETE /identity/v3/projects/3c0c8867869e4009b17356b4af625cfb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:46.187332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:46.202249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.270128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:46.353065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 857/1746] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:46.359611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b758c-6f64-4d15-91c1-0adcbe89aad2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:46.361803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b00937-504b-4b75-aab9-a1359198169f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.362162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b00937-504b-4b75-aab9-a1359198169f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.362452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b00937-504b-4b75-aab9-a1359198169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.381005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:46.384105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.385434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.386025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.391692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.396248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b2b00937-504b-4b75-aab9-a1359198169f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:46.397523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2b00937-504b-4b75-aab9-a1359198169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.522451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 858/1747] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:46.523709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73488bd5-748a-48e3-8c6a-ac1f743aa2a8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.524033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73488bd5-748a-48e3-8c6a-ac1f743aa2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.524460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73488bd5-748a-48e3-8c6a-ac1f743aa2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.552033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:46.562508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-73488bd5-748a-48e3-8c6a-ac1f743aa2a8 None None] MFA Rules not processed for user `aa2438a9459d4e9495d4ca19b6bacec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:46.564539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-73488bd5-748a-48e3-8c6a-ac1f743aa2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.604216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 859/1748] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:46.607242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.610755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.615299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.661121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:46.664607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:46.683667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:46.685742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.745537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 890/1749] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:46.746508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 860/1750] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:46.752354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:46.754519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:46.761028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:46.812327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9af2aa-b8bb-4120-8a1c-86152750139b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:46.841197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:46.853849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.854283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.854727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.856370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:46.859970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b322ac-a243-438f-9cc9-502253756103 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:46.864600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:46.868347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:46.888730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:46.897945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:46.899357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:46.899877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:46.952412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 861/1751] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:46.962963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:46.980121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:46.984880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:46.996656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4713-d737-4958-a445-14740a3a90d9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.008910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.010777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.011419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.012692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.014018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 891/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:47.014679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.020290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.095969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 862/1753] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:46 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:47.101955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:47.128370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8f3ac-0b09-443d-a0e8-41cbf2e2453a service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.305823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.307270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.308035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.308792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.310065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.313969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.373442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 892/1754] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:47 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:36:47.376574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:47.404389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2a3a-3dd7-4218-9c40-82883430470a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.426590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.426590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.426590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.426590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.427387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.427387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.428926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:47.431460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:47.462438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9740-5cfe-4c1c-9f12-8d6b85d024b4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.485629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.488106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 863/1755] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:47.488463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.489691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.490048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] PATH_INFO: `/v3/projects/32517d18e03b448f81c0cfe9eb5cd100` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.494115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:47.497708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32517d18e03b448f81c0cfe9eb5cd100)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.504320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.530839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c38f1-c959-4bcc-92c7-f81e2099e40e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.545894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.549283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.550965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.551639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.554742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.558736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.599078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 864/1756] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:47.606267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:47.647974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0705b6-5b2a-438d-8f00-444ff9e886b6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:47.661942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32517d18e03b448f81c0cfe9eb5cd100', 'request_id': 'req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:47.674791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32517d18e03b448f81c0cfe9eb5cd100', 'request_id': 'req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:47.686717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32517d18e03b448f81c0cfe9eb5cd100', 'request_id': 'req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:47.719739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] The token cache is being invalidate because project 32517d18e03b448f81c0cfe9eb5cd100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:47.720488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:47.725862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-eb13cda4-2fb4-4f68-8bd6-98e960b7ec78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:47.733751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 893/1757] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:47 2022] DELETE /identity/v3/projects/32517d18e03b448f81c0cfe9eb5cd100 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:36:47.754368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91631026-3efa-4390-9fd0-61a955e656a9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.756529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91631026-3efa-4390-9fd0-61a955e656a9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.759309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91631026-3efa-4390-9fd0-61a955e656a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.773687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:47.774721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.775071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.775418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] PATH_INFO: `/v3/projects/793d6eb14dd640b4bfc4b6576ae2a287/users/61f1caa43c1e49e1aa1dd937d52a14a1/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.797175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-91631026-3efa-4390-9fd0-61a955e656a9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:47.800907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-91631026-3efa-4390-9fd0-61a955e656a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:47.813407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793d6eb14dd640b4bfc4b6576ae2a287, user_id=61f1caa43c1e49e1aa1dd937d52a14a1, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:47.816357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918af153-fe87-451a-9f69-e5fa1d88ba70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:47.822936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:47.826751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 894/1758] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:47.836308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.838605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.840943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.846796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 865/1759] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:47 2022] PUT /identity/v3/projects/793d6eb14dd640b4bfc4b6576ae2a287/users/61f1caa43c1e49e1aa1dd937d52a14a1/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:47.858829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:47.859204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:47.859552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:47.910536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:47.914055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:47.937927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] MFA Rules not processed for user `61f1caa43c1e49e1aa1dd937d52a14a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:47.939379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:48.002229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:48.069730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 895/1760] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:48.081075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.104441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12863a2f-a89e-4bb8-a815-b400d59b306f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.114698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.116984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.117522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.118375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.123951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:48.124964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.132407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.151619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 896/1761] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:48.156021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.171331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f31df9-ae7b-4879-85b4-42176f71b82d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.178353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.179528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.180345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.180826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.181730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.185691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.206582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 866/1762] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:36:48.215178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.216975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:48.231881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 897/1763] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:36:48.241034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.276690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fa984-f90d-46aa-b93b-e4e403ee31b4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.532576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.534028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.534533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.534955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.535966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.538069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.538326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7c1a3-ff68-420c-86a5-60aaf4932e55 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.554053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.555333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.555792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.556229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.557064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.560600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.593540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 867/1764] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:48.594788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.598485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:48.622490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 898/1765] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:48 2022] POST /identity/v3/users => generated 292 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:48.623501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:48.625678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:48.657321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0712b-e03b-46af-93c3-2b85f613a390 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.846392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.848979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.850850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.851390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] PATH_INFO: `/v3/projects/56089dea304149f4812013484a62fc32` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.855289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db408b8-b03e-461d-8cf0-d1f0ca08a3de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:48.856941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56089dea304149f4812013484a62fc32)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.864218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.884777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:48.884777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:48.884777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:48.884777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] PATH_INFO: `/v3/users/587c639f761a4180aa0e1e86fbfb4272` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:48.889795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=587c639f761a4180aa0e1e86fbfb4272)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:48.891790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:48.930960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:48.948388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:48.991412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56089dea304149f4812013484a62fc32', 'request_id': 'req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.011169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56089dea304149f4812013484a62fc32', 'request_id': 'req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.030131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56089dea304149f4812013484a62fc32', 'request_id': 'req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.046945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587c639f761a4180aa0e1e86fbfb4272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.052488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587c639f761a4180aa0e1e86fbfb4272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.062372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] The token cache is being invalidate because project 56089dea304149f4812013484a62fc32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:49.063633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.065799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.073638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 899/1766] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:48 2022] DELETE /identity/v3/projects/56089dea304149f4812013484a62fc32 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:49.075374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:49.078613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.104911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587c639f761a4180aa0e1e86fbfb4272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.140431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587c639f761a4180aa0e1e86fbfb4272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.143376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587c639f761a4180aa0e1e86fbfb4272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.180921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 868/1767] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:48 2022] DELETE /identity/v3/users/587c639f761a4180aa0e1e86fbfb4272 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:49.181818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:49.183287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.254639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb97a-c1e1-454e-82f2-6e692f6f7d46 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:49.401733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287e8c-755d-4220-a3dd-140efcc3eb7d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:49.416179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:49.418926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.419321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.439665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.441874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:49.443367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370ca3-a45a-4c6e-841d-dacb1ed3a96a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:49.446875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:49.447637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.447964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.448246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.448959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:49.457830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:49.481613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 869/1768] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:49.493468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2e3d-0dcc-47e1-8bfa-a1a656c116cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.493767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2e3d-0dcc-47e1-8bfa-a1a656c116cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.496426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2e3d-0dcc-47e1-8bfa-a1a656c116cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.538100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 900/1769] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:49 2022] POST /identity/v3/users => generated 292 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:49.544736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:49.555070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.578952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88bf2e3d-0dcc-47e1-8bfa-a1a656c116cc None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:49.582654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88bf2e3d-0dcc-47e1-8bfa-a1a656c116cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.590445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f179a3-1e61-43ef-a068-ec0f304ea150 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:49.608781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:49.609761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.610166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.610495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] PATH_INFO: `/v3/users/40ae05d9f5924822b6a3b704b311bbc2` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.614894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ae05d9f5924822b6a3b704b311bbc2)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:49.619690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:49.766451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ae05d9f5924822b6a3b704b311bbc2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.768220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 870/1770] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Jul 20 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:49.771874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.772212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.772856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.789759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:49.799130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ae05d9f5924822b6a3b704b311bbc2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.801339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ae05d9f5924822b6a3b704b311bbc2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.802900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ae05d9f5924822b6a3b704b311bbc2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.830503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fe44f5ab-b634-4781-b190-876eb2aae486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ae05d9f5924822b6a3b704b311bbc2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:49.835550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:49.841027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.858754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 901/1771] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:36:49 2022] DELETE /identity/v3/users/40ae05d9f5924822b6a3b704b311bbc2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:49.861403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:49.861847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:49.864133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:49.864725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 871/1772] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:49.867395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:49.868237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 902/1773] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jul 20 15:36:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:49.869875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.870297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:49.874938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:49.914966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14106e12-3a41-4020-9d1e-f95ff76c954d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:49.974139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:50.049156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.050166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.052211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.052931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.054367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.062222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde20616-062e-47cc-9f22-f6ca4da4f17e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.094313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 872/1774] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:50.099195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.101892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.102558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.176489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:50.181174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872d57d-3122-4fa3-ab27-a83bdca034e3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.185398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:50.210692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.211500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.211713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.217688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.222849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.233699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.265208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 903/1775] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:50.267410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:50.274472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:50.448840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 873/1776] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:50.450676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:50.455087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:50.474043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f752f-8153-4751-bcfa-305aba561887 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.485476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b79f-bfe5-4102-ab64-782f6e3e7e00 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.492689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.494914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.495202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.495485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.498615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.502634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.503267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.506893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.516340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:50.540504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.545796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.546284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.548851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.570784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 874/1777] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:50.572747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:50.581467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 904/1778] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:50.583396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:50.601003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a918ed-6241-447d-9561-f2cf4eecf4ef service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.624267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.625416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.625867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.626429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc42cd-75ad-444b-8ef8-693017122ae9 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.628139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] PATH_INFO: `/v3/projects/c2b7bd39b3d14e18b3f363a3ec48f3c6/users/ca5eae5fd4de4d969093b59b09949eb5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.643020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:50.643712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:50.644040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:50.644509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:50.645439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.648836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.682961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 905/1779] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:50.684470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:50.692918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b7bd39b3d14e18b3f363a3ec48f3c6, user_id=ca5eae5fd4de4d969093b59b09949eb5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:50.696280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:50.716765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f082a-d7ad-480e-ab1b-f31f9105e904 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:50.726556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 875/1780] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:50 2022] PUT /identity/v3/projects/c2b7bd39b3d14e18b3f363a3ec48f3c6/users/ca5eae5fd4de4d969093b59b09949eb5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:50.727710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:51.012542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:51.013247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:51.013534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:51.013836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:51.045781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75866d5-bb9a-4ecb-82d7-8bc016a6d1bf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:51.048303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:51.051639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8421e25-d718-48fd-842a-4d14ec87a744 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:51.069043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:51.070436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:51.070725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:51.071009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] PATH_INFO: `/v3/projects/2b8d385f4c2d442285173b7ce74d7816/users/b15bf9424671466a977d9b5a1ebc6fba/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:51.083996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 906/1781] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:51.085446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:51.085737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:51.086017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:51.122756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8d385f4c2d442285173b7ce74d7816, user_id=b15bf9424671466a977d9b5a1ebc6fba, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:51.125557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8e6b-d8fd-4403-affd-0586c1915db2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:51.135073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:51.162340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 876/1782] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:50 2022] PUT /identity/v3/projects/2b8d385f4c2d442285173b7ce74d7816/users/b15bf9424671466a977d9b5a1ebc6fba/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:51.163976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] MFA Rules not processed for user `ca5eae5fd4de4d969093b59b09949eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:51.165223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:51.176886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:51.177200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:51.177457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:51.293687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] MFA Rules not processed for user `b15bf9424671466a977d9b5a1ebc6fba`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:51.294961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:51.329853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:51.524128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 907/1783] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:51.529136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:51.638505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 877/1784] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:51.642966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:51.756215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd217c7-2162-4b81-9d99-91be473e18dd None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:51.769464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:51.770159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:51.770449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:51.770730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:51.772214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:51.775924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:51.841734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9b1cf-c355-49ca-b499-2c111caeabd8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:51.846809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 908/1785] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:51 2022] POST /identity/v3/projects => generated 379 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:51.848414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:52.002963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a12b30-d536-46ec-9acc-ef2e2e0c5942 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:52.006018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.008563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.008925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.009043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.009393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.009705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.010001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.010280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:52.010389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.011038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:52.013914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:52.014068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:52.061633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 909/1786] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:51 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:52.075862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:52.080282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:52.108562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0660fb-a8f4-46bd-9652-4592a39e90fd admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:52.122080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.123476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.124023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.124365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] PATH_INFO: `/v3/projects/af2b2a05a6984a8ab7c35ffe4ef3be1b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.124774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 878/1787] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:51 2022] POST /identity/v3/projects => generated 387 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:52.126976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:52.128600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2b2a05a6984a8ab7c35ffe4ef3be1b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:52.137776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:52.156475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:52.229456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2b2a05a6984a8ab7c35ffe4ef3be1b', 'request_id': 'req-269227e3-07a0-45ea-84b4-1d1b467a1a13'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:52.240562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2b2a05a6984a8ab7c35ffe4ef3be1b', 'request_id': 'req-269227e3-07a0-45ea-84b4-1d1b467a1a13'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:52.257577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2b2a05a6984a8ab7c35ffe4ef3be1b', 'request_id': 'req-269227e3-07a0-45ea-84b4-1d1b467a1a13'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:52.286451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:52.318562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] The token cache is being invalidate because project af2b2a05a6984a8ab7c35ffe4ef3be1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:52.318833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:52.324768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:52.329300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 910/1788] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:52 2022] DELETE /identity/v3/projects/af2b2a05a6984a8ab7c35ffe4ef3be1b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:52.330144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:52.331641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:52.368364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751ce57-df70-4dfb-8804-e2b3ce6c69fa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:52.554936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.555641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.559359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.559809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.562171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:52.568045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4efa8-936b-4084-845d-2a8dda62f934 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:52.613676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 879/1789] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:52.613676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.613676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.613676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.640516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269227e3-07a0-45ea-84b4-1d1b467a1a13 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:52.657145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] MFA Rules not processed for user `aa2438a9459d4e9495d4ca19b6bacec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:52.658872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:52.664059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.669015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.669311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.669683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.670407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:52.680793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b0cb4-8ae6-4de3-8743-aa690669d0a8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:52.812796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 911/1790] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:52 2022] POST /identity/v3/users => generated 300 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:52.814165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.818489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.818941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.880233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 880/1791] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:52.883250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:52.886189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:52.923965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed09bc-33ed-49ac-9fad-89f2929519ec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:52.941050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:52.942148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:52.942591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:52.942978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] PATH_INFO: `/v3/projects/1c14368888b84f4b871dd5e0ef2922ba/users/932cd8194442433996e01d5285e6a65b/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:52.946672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:52.948220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:52.985606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 912/1792] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:52.985606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:53.013272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c14368888b84f4b871dd5e0ef2922ba, user_id=932cd8194442433996e01d5285e6a65b, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:53.018733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7336a-f900-4fac-8d61-d7e22761dd83 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:53.022532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:53.040417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:53.042307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.042467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.044872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.045668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:53.051649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214ff1e-f6b9-478d-bb79-f44c0e9b9190 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:53.052405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 881/1793] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:52 2022] PUT /identity/v3/projects/1c14368888b84f4b871dd5e0ef2922ba/users/932cd8194442433996e01d5285e6a65b/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:53.054824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:53.060918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:53.086678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 913/1794] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:53.089478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.090009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.090122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.135413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:53.162094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:53.426180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928a3d5-5c79-4e00-befe-fc0e45ab3748 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:53.435038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:53.436428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.436712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.436986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] PATH_INFO: `/v3/projects/5c17597384fc402bbce04b723d4cfef8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.450126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c17597384fc402bbce04b723d4cfef8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:53.455016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:53.478064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:53.530624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 914/1795] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:53.531622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:53.567224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c039f74-c93f-4f62-b6ae-b3877dc8127d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:53.596373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:53.624348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c17597384fc402bbce04b723d4cfef8', 'request_id': 'req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:53.646736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c17597384fc402bbce04b723d4cfef8', 'request_id': 'req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:53.670883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c17597384fc402bbce04b723d4cfef8', 'request_id': 'req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:53.728231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] The token cache is being invalidate because project 5c17597384fc402bbce04b723d4cfef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:36:53.728428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:53.732797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:36:53.736742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 882/1796] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:36:53 2022] DELETE /identity/v3/projects/5c17597384fc402bbce04b723d4cfef8 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:53.738440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:53.741048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:53.813631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:53.814938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.815459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.822378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.823518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:53.827149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:53.868144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 915/1797] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:53.869292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:53.872901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:53.908953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccc71d-4f4e-4f27-853b-7fa4a2f19d2a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:53.914682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:53.920307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.924434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.926337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.930290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:53.930729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:53.932544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8270a-4c15-4e0d-8710-60269c977e72 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:53.966300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 916/1798] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:53.968767 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.969167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.969794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:53.972469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f89947-ce0f-4bdb-bdd6-9ad4b2065748 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:53.982422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:53.995331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:53.996235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:53.999708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] PATH_INFO: `/v3/projects/1b4305c980174331bc51d31b6b113f97/users/8ea6accfd5eb46fa9cff20b497a10479/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.044109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:54.061126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:54.083727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4305c980174331bc51d31b6b113f97, user_id=8ea6accfd5eb46fa9cff20b497a10479, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.086573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.108099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 917/1799] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:54.109094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.111931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:54.131441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 883/1800] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:53 2022] PUT /identity/v3/projects/1b4305c980174331bc51d31b6b113f97/users/8ea6accfd5eb46fa9cff20b497a10479/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:54.133408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.147840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa688a-f782-4286-acbb-25b7ce64068a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:54.256942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:54.258451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.258848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.259706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] PATH_INFO: `/v3/projects/1c14368888b84f4b871dd5e0ef2922ba/users/932cd8194442433996e01d5285e6a65b/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.263143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c14368888b84f4b871dd5e0ef2922ba, user_id=932cd8194442433996e01d5285e6a65b, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.266072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22cc08-6b4c-4ce3-b9b2-8015518ac5c9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.279949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 918/1801] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:54 2022] PUT /identity/v3/projects/1c14368888b84f4b871dd5e0ef2922ba/users/932cd8194442433996e01d5285e6a65b/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:54.292538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9868-a382-4013-a364-793591297faa None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.293010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9868-a382-4013-a364-793591297faa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.293364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9868-a382-4013-a364-793591297faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.315485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a126d6-b9a5-45b8-a566-d509678c9d6b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:54.317630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b9aa9868-a382-4013-a364-793591297faa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:54.319301 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b9aa9868-a382-4013-a364-793591297faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:54.374621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:54.515736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:54.517595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.517893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.518867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.519998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.522810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 919/1802] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:54.528719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa9868-a382-4013-a364-793591297faa None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.532018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.554089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa9868-a382-4013-a364-793591297faa None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:54.568644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:54.569468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.569855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.570344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.574362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.581175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7219e-8974-484b-9004-1671be81ff87 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.605203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 920/1803] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:54.605203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.605203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.605203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.610269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 884/1804] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:54 2022] POST /identity/v3/projects => generated 379 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:54.610269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.661378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:54.692394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c273d54-b65c-484e-a63e-b4d31483569f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:54.699067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] MFA Rules not processed for user `932cd8194442433996e01d5285e6a65b`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:54.700392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:54.700518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:54.701568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.702060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.702539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.703505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:36:54.704223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.706314 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.729680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 885/1805] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:36:54.729680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.740806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88c24d-fd47-4902-a02c-043d26ae3b9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:54.747972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:54.749442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:54.750010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:54.750570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] PATH_INFO: `/v3/projects/1b4305c980174331bc51d31b6b113f97/users/8ea6accfd5eb46fa9cff20b497a10479/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:54.754294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4305c980174331bc51d31b6b113f97, user_id=8ea6accfd5eb46fa9cff20b497a10479, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:54.757421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:54.836733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 886/1806] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:54 2022] PUT /identity/v3/projects/1b4305c980174331bc51d31b6b113f97/users/8ea6accfd5eb46fa9cff20b497a10479/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:54.838819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.927716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 921/1807] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:54.930447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:54.958447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2405c-774a-48c2-a3c4-cbd659eeef8b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:55.055408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceac2f-c948-4b2c-b61e-e99fc00df095 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:55.069163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:55.070773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:55.071973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:55.073673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:55.075920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:55.082009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4f950-7c58-4827-8faa-f9a913102d6c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:55.189699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 887/1808] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:54 2022] POST /identity/v3/users => generated 290 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:55.195255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:55.197150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:55.197545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:55.248020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:55.252285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:55.252587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:55.255211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:55.256150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:55.268350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:55.349347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] MFA Rules not processed for user `8ea6accfd5eb46fa9cff20b497a10479`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:55.355940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 922/1809] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:55.356144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:55.358741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:55.364001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:55.395589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:55.529433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd2fe6-6136-4cf7-9515-b2b045b878c0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:55.541683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:55.543880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:55.544189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:55.544914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:55.597423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:55.726317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:55.735961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 888/1810] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:55.736583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:55.736728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:55.774323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 923/1811] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:55.775136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:55.936640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f2ae9-6d55-420d-86f5-c7e8328891a1 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:55.962149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:55.963174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:55.964779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:55.965557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:55.966612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:55.969092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.067468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 924/1812] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:55 2022] POST /identity/v3/users => generated 291 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:56.071150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.092054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6fc4-7d1d-4138-87ab-5eefc3484e12 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.106320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9f41e-d785-4ca8-aa76-a617fb111219 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.115555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.118068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.118586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.119063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.120155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.133843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.156690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.162905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.163723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.165142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.174124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 889/1813] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:56.174915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.212388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe6ec-a022-4569-b056-bc7606184a00 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.229735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.239916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.240709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.242222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.248619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.249300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.253683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.260092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.292221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 925/1814] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:56.298572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.315445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 890/1815] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:56.323309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.346058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f1d5b-5a61-4c6c-8e60-e4c317e8ff11 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.363947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.364890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.365118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13d223-6122-4686-b6fd-66fa46d72b3f service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.365588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.366066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.367213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.369322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.399920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.400621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.400908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.412070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.412982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.424742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 891/1816] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:56.426496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.444175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.463823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d6ad1-6cdf-4c7f-a773-2754c761ad29 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.484386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 926/1817] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:56.488966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.489153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.489895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.490175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.490449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] PATH_INFO: `/v3/projects/fa01f791c63548969cb2049c7f4396a7/users/b5271ad9891549f3b881b9aa17adeae3/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.523083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa01f791c63548969cb2049c7f4396a7, user_id=b5271ad9891549f3b881b9aa17adeae3, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.523083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe221c4-a7e6-4f67-801d-47fa1c03b3ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.545584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb83465-3b03-45fb-9e69-7e65b85ec974 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.559985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 892/1818] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:56 2022] PUT /identity/v3/projects/fa01f791c63548969cb2049c7f4396a7/users/b5271ad9891549f3b881b9aa17adeae3/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:56.571096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.573175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.574810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.649997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.659537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.660241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.660638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] PATH_INFO: `/v3/projects/a9296f4a79f348cf9a6662c2ba21dab6/users/8488e4e249f24d49a4b1c0d5a051e114/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.690032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] MFA Rules not processed for user `b5271ad9891549f3b881b9aa17adeae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:56.707969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:56.729707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9296f4a79f348cf9a6662c2ba21dab6, user_id=8488e4e249f24d49a4b1c0d5a051e114, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.734299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bc17-8280-42b6-80da-a2f7a0de7a3d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.798940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 927/1819] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:56 2022] PUT /identity/v3/projects/a9296f4a79f348cf9a6662c2ba21dab6/users/8488e4e249f24d49a4b1c0d5a051e114/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:56.811448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.815618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.816726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.890059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:56.917085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] MFA Rules not processed for user `8488e4e249f24d49a4b1c0d5a051e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:56.920006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:56.937503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:56.965908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 893/1820] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:56.965908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.967963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:57.052122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d14c0-6b65-4514-a0ec-354eb0ef1f06 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.107650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 928/1821] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:57.113556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.135930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.137529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.138179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.138745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.141483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:57.229933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.231989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.252058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 894/1822] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:57.258645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.273514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c3c55-715c-4d39-84e7-0c3e08a3ea97 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.284253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.284934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.285223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.285500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.286203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.290417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.306571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf1fcf-68cd-4f4f-9106-343737757b3c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.335129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.338245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.338369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.338570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.339236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 929/1823] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:57 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:57.340481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.341212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.348813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.440972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 895/1824] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:36:57 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:57.442914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.622061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:57.646159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ea9a8-0fca-45df-956f-09141e5088ae admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.794610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6b942-7a75-4109-8480-698613ba315a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.802420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.803529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.804341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.804815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.805949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.808676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378aff0-5019-4a27-b512-31b24e6dea42 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.817755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:57.906153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 896/1825] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:57 2022] POST /identity/v3/users => generated 297 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:57.909586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e3987-c3a2-4670-8e46-8a73ae80a66c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.910548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e3987-c3a2-4670-8e46-8a73ae80a66c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.911173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e3987-c3a2-4670-8e46-8a73ae80a66c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.913675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 897/1826] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:36:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:36:57.915245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e3987-c3a2-4670-8e46-8a73ae80a66c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.929545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.934994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.935387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.935577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.936317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.938379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.989857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e3987-c3a2-4670-8e46-8a73ae80a66c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.023995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 930/1827] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:36:57 2022] POST /identity/v3/users => generated 296 bytes in 680 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:36:58.024950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.027092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.029150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.031948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.032707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.057379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.063713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.069561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff186536-3fa9-49a6-aed1-738454c89fc6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.086986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.087691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.088016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.088679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.089060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.091261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.109989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 898/1828] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:58.111283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.137958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 931/1829] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:58.140831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920e97-deca-4811-9016-eb7eb894b05b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.148286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.173814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.173814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.173814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.173814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] PATH_INFO: `/v3/projects/e6bf50ee0542429294868c50cfd97b8a/users/ba4f2faf24db44f4829294a2bc6843fb/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.188472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53960e7-98c5-45f8-a072-0e608e70d9dd admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.209180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bf50ee0542429294868c50cfd97b8a, user_id=ba4f2faf24db44f4829294a2bc6843fb, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.221709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.223192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.223516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.225357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] PATH_INFO: `/v3/projects/5176a46e508a483389aaae2f5970d8f6/users/a93f7ff50b6449d99b008fab52ff6907/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.231375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.281621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5176a46e508a483389aaae2f5970d8f6, user_id=a93f7ff50b6449d99b008fab52ff6907, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.287375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 899/1830] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:58 2022] PUT /identity/v3/projects/e6bf50ee0542429294868c50cfd97b8a/users/ba4f2faf24db44f4829294a2bc6843fb/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:58.291396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.291696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.352559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 932/1831] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:58 2022] PUT /identity/v3/projects/5176a46e508a483389aaae2f5970d8f6/users/a93f7ff50b6449d99b008fab52ff6907/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:58.355681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.520638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0733-2179-4027-8ffc-1fddf92ef93a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.527736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.528976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.529446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.530006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.531125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.533407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.552119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 933/1832] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:58.560615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.608045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816682-14c2-4a27-adc7-5c28bddcd8c2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.622194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.623182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.623465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.623794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] PATH_INFO: `/v3/projects/5176a46e508a483389aaae2f5970d8f6/users/a93f7ff50b6449d99b008fab52ff6907/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.629332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961efc7-44d2-4c8b-bdf6-cc17e55e4a70 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.641750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5176a46e508a483389aaae2f5970d8f6, user_id=a93f7ff50b6449d99b008fab52ff6907, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.652954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.657733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.658522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.658884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.659211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.660152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.661831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.684247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 934/1833] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:58 2022] PUT /identity/v3/projects/5176a46e508a483389aaae2f5970d8f6/users/a93f7ff50b6449d99b008fab52ff6907/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:58.685256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.692926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 900/1834] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:36:58.699874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:58.734076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05cab-6944-46ba-8d04-8c859b012cba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.842655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a5bc-84c7-4021-8aed-d14b00a05ccc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:58.846920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.853479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.853950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.855806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.874818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:58.934717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:58.938647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.940424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.940777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] PATH_INFO: `/v3/projects/e6bf50ee0542429294868c50cfd97b8a/users/ba4f2faf24db44f4829294a2bc6843fb/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.947615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bf50ee0542429294868c50cfd97b8a, user_id=ba4f2faf24db44f4829294a2bc6843fb, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:58.952264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb8df4-4204-461b-b67d-3f53c1e73773 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:58.991357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 901/1835] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:36:58 2022] PUT /identity/v3/projects/e6bf50ee0542429294868c50cfd97b8a/users/ba4f2faf24db44f4829294a2bc6843fb/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:36:58.991357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a35e-0b53-4e78-a681-6ea951404d92 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:58.991357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a35e-0b53-4e78-a681-6ea951404d92 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:58.991357 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712a35e-0b53-4e78-a681-6ea951404d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:58.992020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:59.047052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:59.049005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee7465-0f2b-48d9-8fe4-8c40bc253661 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:59.084376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 935/1836] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:59.088102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:59.088225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5712a35e-0b53-4e78-a681-6ea951404d92 None None] MFA Rules not processed for user `a93f7ff50b6449d99b008fab52ff6907`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:59.089269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:59.089394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5712a35e-0b53-4e78-a681-6ea951404d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:59.089861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:59.104254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:59.208982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] MFA Rules not processed for user `ba4f2faf24db44f4829294a2bc6843fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:59.211156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:59.340565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 902/1837] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:59.343689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:59.344047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:59.344334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:59.377863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:59.382246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:59.471974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 936/1838] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:59.473782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:59.697303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3920-66b0-40d6-b79f-837e591a2017 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:59.704064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:59.704818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:59.705179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:59.705578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:59.717115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 903/1839] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:59.726996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:59.729252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:59.750159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:59.752447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:59.766060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 937/1840] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:59.775195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b411e9-9fb3-4c88-b084-c5b543e27957 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:59.787367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:59.788495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:59.788947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:59.792241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:59.853630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:59.855788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:59.867210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:59.881798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e82969-58b4-4095-b98e-79f6ea696bb1 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:59.893361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 904/1841] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:59.893976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:59.894761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:59.895050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:59.895333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:59.934643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:59.936141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:59.986202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 938/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:00.387214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:00.402386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dff40-2e6e-4d78-9728-b954bcee1bf1 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:00.409503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:00.410998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:00.411554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:00.412029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:00.593941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:00.596294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8db8a-92c3-4a73-a6c1-eb90b47d4d26 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:00.627974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 905/1843] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:01.114733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:01.179039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:01.201293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe34cdf-1bac-4b0f-8336-be3a2b86d04a service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:01.209015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:01.210065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.210291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.210764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] PATH_INFO: `/v3/users/b5271ad9891549f3b881b9aa17adeae3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.212187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5271ad9891549f3b881b9aa17adeae3)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:01.214259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:01.291864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5271ad9891549f3b881b9aa17adeae3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.312918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5271ad9891549f3b881b9aa17adeae3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.313873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5271ad9891549f3b881b9aa17adeae3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.314461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5271ad9891549f3b881b9aa17adeae3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.325810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-2c7636e0-aa22-4102-a6c2-3f978cdd9339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5271ad9891549f3b881b9aa17adeae3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.364995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 939/1844] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:01 2022] DELETE /identity/v3/users/b5271ad9891549f3b881b9aa17adeae3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:01.369264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.369567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.369821 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.393493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:01.396673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.433644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:01.498205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a797-12a8-41b2-80bc-65738ef7e4b7 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.498771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a797-12a8-41b2-80bc-65738ef7e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.499060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a797-12a8-41b2-80bc-65738ef7e4b7 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.500647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 940/1845] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:37:01.507487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.508240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.508740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.535929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:01.538174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.553004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 906/1846] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:01.566932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:01.568474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.641402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 941/1847] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:01.642842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b28c5-9ce6-4305-afa7-ff0493723e0b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:01.646912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:01.647829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.648272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.648693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.653203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:01.654535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.659533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:01.661209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:01.667968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157dc0d-8448-4499-a0da-e982d6337bbf None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:01.671061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 907/1848] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:01.673653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:01.674302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.674583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.674856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.684449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:01.686029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c7840-9530-465a-bfca-c0d1e739574b service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:01.696352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 942/1849] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:01.788878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:01.790285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.803542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385faa9a-0ba6-4a51-8a04-3a6560931812 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:01.808807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:01.809697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.809978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.810251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] PATH_INFO: `/v3/users/ca5eae5fd4de4d969093b59b09949eb5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.811856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5eae5fd4de4d969093b59b09949eb5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:01.813391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:01.862600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5eae5fd4de4d969093b59b09949eb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.863279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5eae5fd4de4d969093b59b09949eb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.876952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5eae5fd4de4d969093b59b09949eb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.883931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5eae5fd4de4d969093b59b09949eb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.884658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5eae5fd4de4d969093b59b09949eb5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:01.898000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 908/1850] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:01 2022] DELETE /identity/v3/users/ca5eae5fd4de4d969093b59b09949eb5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:01.903427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:01.903885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:01.904207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:01.925818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:01.927024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:01.979117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:01.981063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:01.982637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:02.034233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 943/1851] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:02.047246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:02.050845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:02.052656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:02.097008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda9f3-12d2-491d-b76d-7a21cf08cc1e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:02.106444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:02.106976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:02.107263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:02.108903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] PATH_INFO: `/v3/projects/fa01f791c63548969cb2049c7f4396a7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:02.112070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa01f791c63548969cb2049c7f4396a7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:02.116523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:02.170276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d116b-3d31-41c3-a33c-71f725357282 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:02.181467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:02.182137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:02.182416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:02.184604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:02.209379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:02.211080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:02.213503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa01f791c63548969cb2049c7f4396a7', 'request_id': 'req-32244dfe-a170-4b24-9428-62828de7ef5f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.225466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 944/1852] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:02.230090 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa01f791c63548969cb2049c7f4396a7', 'request_id': 'req-32244dfe-a170-4b24-9428-62828de7ef5f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.239866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa01f791c63548969cb2049c7f4396a7', 'request_id': 'req-32244dfe-a170-4b24-9428-62828de7ef5f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.256667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] The token cache is being invalidate because project fa01f791c63548969cb2049c7f4396a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:02.257057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.260955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.263533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 909/1853] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:01 2022] DELETE /identity/v3/projects/fa01f791c63548969cb2049c7f4396a7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:02.267199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:02.278569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cde83c-e984-4eee-9e38-426d1538eb16 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:02.388104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:02.390115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:02.390115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:02.390115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] PATH_INFO: `/v3/users/a93f7ff50b6449d99b008fab52ff6907` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:02.390998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93f7ff50b6449d99b008fab52ff6907)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:02.393216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:02.477822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93f7ff50b6449d99b008fab52ff6907'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.502934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93f7ff50b6449d99b008fab52ff6907'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.504790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93f7ff50b6449d99b008fab52ff6907'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.505854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93f7ff50b6449d99b008fab52ff6907'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.514492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93f7ff50b6449d99b008fab52ff6907'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.533755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 945/1854] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:02 2022] DELETE /identity/v3/users/a93f7ff50b6449d99b008fab52ff6907 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:02.622409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:02.624941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:02.648515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32244dfe-a170-4b24-9428-62828de7ef5f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:02.731225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:02.732689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:02.732867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:02.733367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] PATH_INFO: `/v3/projects/c2b7bd39b3d14e18b3f363a3ec48f3c6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:02.734870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b7bd39b3d14e18b3f363a3ec48f3c6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:02.737365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:02.827524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b7bd39b3d14e18b3f363a3ec48f3c6', 'request_id': 'req-d3ed775b-e363-4706-b476-5309d233d299'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.837671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b7bd39b3d14e18b3f363a3ec48f3c6', 'request_id': 'req-d3ed775b-e363-4706-b476-5309d233d299'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.847632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b7bd39b3d14e18b3f363a3ec48f3c6', 'request_id': 'req-d3ed775b-e363-4706-b476-5309d233d299'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.852825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:02.884344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] The token cache is being invalidate because project c2b7bd39b3d14e18b3f363a3ec48f3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:02.884711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.885524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:02.888823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 910/1855] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:02 2022] DELETE /identity/v3/projects/c2b7bd39b3d14e18b3f363a3ec48f3c6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:02.894135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:02.895631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:02.913123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c998-c324-4a7c-bd88-a3519b09b6fe admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:02.923621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:03.016109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:03.017097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:03.017449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:03.017792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] PATH_INFO: `/v3/users/932cd8194442433996e01d5285e6a65b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:03.019093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] RBAC: Authorizing `identity:delete_user(user_id=932cd8194442433996e01d5285e6a65b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:03.021224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:03.094545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932cd8194442433996e01d5285e6a65b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:03.110898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932cd8194442433996e01d5285e6a65b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:03.112130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932cd8194442433996e01d5285e6a65b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:03.112671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932cd8194442433996e01d5285e6a65b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:03.122573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dd8eb4e5-5748-4b7e-807b-161bef06bcac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932cd8194442433996e01d5285e6a65b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:03.146688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 946/1856] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:02 2022] DELETE /identity/v3/users/932cd8194442433996e01d5285e6a65b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:03.632779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:03.634633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:03.653312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed775b-e363-4706-b476-5309d233d299 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:03.869443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:03.873236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:03.873236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:03.873236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] PATH_INFO: `/v3/projects/5176a46e508a483389aaae2f5970d8f6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:03.885152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5176a46e508a483389aaae2f5970d8f6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:03.888340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.030189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5176a46e508a483389aaae2f5970d8f6', 'request_id': 'req-125abec9-d99a-4b7f-a1c2-42543b8b6da4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.057861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5176a46e508a483389aaae2f5970d8f6', 'request_id': 'req-125abec9-d99a-4b7f-a1c2-42543b8b6da4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.077637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5176a46e508a483389aaae2f5970d8f6', 'request_id': 'req-125abec9-d99a-4b7f-a1c2-42543b8b6da4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.099325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] The token cache is being invalidate because project 5176a46e508a483389aaae2f5970d8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:04.099726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.100539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-125abec9-d99a-4b7f-a1c2-42543b8b6da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.108682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.109261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.110068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.118150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 911/1857] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:03 2022] DELETE /identity/v3/projects/5176a46e508a483389aaae2f5970d8f6 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:04.133874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:04.135229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.218378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.218937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.219380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.223445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 947/1858] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:04.241849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:04.243403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.256058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.258869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.271116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 912/1859] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:04.280448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb95a97-6dea-4317-a240-aefef299e2f3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.283169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.286221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.287183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.287599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.287910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] PATH_INFO: `/v3/projects/1c14368888b84f4b871dd5e0ef2922ba` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.289613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c14368888b84f4b871dd5e0ef2922ba)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.298983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.315663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7e06e-3a31-436b-862a-89bd3e76c0dc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.329810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.330236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.330519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.330795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.331534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:04.331985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.333664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.355297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:04.365412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 913/1860] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:04.369591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.386278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077e57c-ccb5-47b7-bb9d-b65229a4c952 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.415292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c14368888b84f4b871dd5e0ef2922ba', 'request_id': 'req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.440679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c14368888b84f4b871dd5e0ef2922ba', 'request_id': 'req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.454861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c14368888b84f4b871dd5e0ef2922ba', 'request_id': 'req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.488215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] The token cache is being invalidate because project 1c14368888b84f4b871dd5e0ef2922ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:04.488583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.490556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-670eb8ea-46a2-4cdb-a724-5c0faf4d8c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:04.495705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 948/1861] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:04 2022] DELETE /identity/v3/projects/1c14368888b84f4b871dd5e0ef2922ba => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:04.503920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f914bf-8466-4820-95ed-ab2638b641eb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.503920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f914bf-8466-4820-95ed-ab2638b641eb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.503920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f914bf-8466-4820-95ed-ab2638b641eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.526667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-66f914bf-8466-4820-95ed-ab2638b641eb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:04.528272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-66f914bf-8466-4820-95ed-ab2638b641eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.556419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.557111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.557395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.557672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.558927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.561487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.591814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:04.616466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 914/1862] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:04.617510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.619268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.624898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 949/1863] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:04.632250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.632600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.632888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.658964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:04.660180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.713346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786e9b3-91b0-4c8c-9e6d-d3e0ff0d5e96 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.718520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.719213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.719498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.719840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.720513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.722450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.741416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 950/1864] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:04.743116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.755885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 915/1865] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:04.756282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2e507-805e-4a53-a7af-7f4e5a79caef None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.757505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.761623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.763862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.763862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.764085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.765136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:04.765455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.767462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.770650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ff9ee-5aee-4615-83bd-5b3c3f83ca08 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.776003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.777076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.777647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.778303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.779214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.781021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.785475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 951/1866] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:04.786886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.797043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe97495-f0ae-4d9e-9a2e-a4d8cf5ca74e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.801191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 916/1867] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:04.805445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.807876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.809667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.810159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.810441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.811141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.813830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.819672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ef32c-7e67-4b61-b82d-3fd67130fd35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.825345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.826673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.827028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.827391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.828283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.830081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.844974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 917/1868] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:04.849216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.859709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 952/1869] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:04.860729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:04.870003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820ebfd-bb0c-4c01-a150-7fcf70fe84c9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.884825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:04.956142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.956324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7fa42-8fcf-48e3-94be-cea5337a81fd admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.957055 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.957366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.957641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] PATH_INFO: `/v3/projects/1f5e5c49b19540a085ecc519b672cb7e/users/0b412e35f2f34ec89176a48da1bb85b8/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.961156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.961838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.962121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.962401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.963092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.965085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.979684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5e5c49b19540a085ecc519b672cb7e, user_id=0b412e35f2f34ec89176a48da1bb85b8, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.983116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0282aa1-a1da-4049-a2f7-c2b7d88c79a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.990604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 953/1870] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:04 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:04.994754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.003690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 918/1871] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:04 2022] PUT /identity/v3/projects/1f5e5c49b19540a085ecc519b672cb7e/users/0b412e35f2f34ec89176a48da1bb85b8/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:05.007455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5560c-0b27-4b55-aedf-cf3039270781 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.014458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.014458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.014458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.043566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] MFA Rules not processed for user `0b412e35f2f34ec89176a48da1bb85b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:05.044888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:05.070955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.071932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.072282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.072593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.073330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.075146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.078422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:05.089427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 954/1872] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:05.090264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.100333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871811e4-b022-4c1f-8e84-ed64a5ad57db admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.105051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.106243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.106532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.106810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.107574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.109316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.122076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 955/1873] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:05.122348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 919/1874] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:05.126749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.129687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.136936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554eaf-6a74-4569-99bb-ee0a16be6870 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.141298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388632-d1b9-4832-a3db-bb62e337f2ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.141740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.142749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.143036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.143438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] PATH_INFO: `/v3/projects/88ff64a7aa77424c8da614400df6bffb/users/a6359f6129cc42ecb571da03ee2e072f/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.146381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.147481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.147481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.147850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.148597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.150716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154df89c-c741-4c20-ab8f-831dbd39f577 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.159081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ff64a7aa77424c8da614400df6bffb, user_id=a6359f6129cc42ecb571da03ee2e072f, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.161847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.176280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 956/1875] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:05 2022] PUT /identity/v3/projects/88ff64a7aa77424c8da614400df6bffb/users/a6359f6129cc42ecb571da03ee2e072f/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:05.184047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 920/1876] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:05.185508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.187917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.188264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.189171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.233861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] MFA Rules not processed for user `a6359f6129cc42ecb571da03ee2e072f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:05.235073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:05.255677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578126ea-1031-4d1c-8f52-3163908da0a4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.264553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:05.301502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 921/1877] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:05.305835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.313829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.314504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.314791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.315074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.315783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.317714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.340657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 957/1878] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:05.347693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.359168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb3053-b87a-4a96-b6de-aef4e4b7da0a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.364735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.365410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.365697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.365976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.366737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.368571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.372519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed41a0d-60f8-4b76-b694-796a3df7aa0e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.377189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.377975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.378259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.378534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.379134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 958/1879] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:05.379261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.381137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.383226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.392658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cb4ad-74a9-4d78-8ed8-1be469752f31 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.397284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.398415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.398829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.399235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] PATH_INFO: `/v3/projects/7efc76c2b13340f0a7d779d75dff96fa/users/e4b52c0d94d144e4b5bde97074ae1468/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.410824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 922/1880] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:05.411867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.415171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efc76c2b13340f0a7d779d75dff96fa, user_id=e4b52c0d94d144e4b5bde97074ae1468, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.418600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.437009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 959/1881] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:05 2022] PUT /identity/v3/projects/7efc76c2b13340f0a7d779d75dff96fa/users/e4b52c0d94d144e4b5bde97074ae1468/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:05.441062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.446434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:05.484073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e254708-3f27-4c6c-adbe-aaf1275d3e76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.505503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20694d-fc0a-457a-9379-3e3a0be82fb8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.510085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.510748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.511035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.511310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.512618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.514205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.524015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 960/1882] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:05.528403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.540038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f1ded-b159-4074-9f84-8e5b4d587125 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.544511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.544919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.545196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.545484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.545763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.546461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.547171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.547472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.547752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] PATH_INFO: `/v3/projects/7efc76c2b13340f0a7d779d75dff96fa/users/e4b52c0d94d144e4b5bde97074ae1468/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.548575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501898e-3ee5-4d48-b3cc-b27d675063a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.550381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efc76c2b13340f0a7d779d75dff96fa, user_id=e4b52c0d94d144e4b5bde97074ae1468, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.553125 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.565808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 961/1883] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:05 2022] PUT /identity/v3/projects/7efc76c2b13340f0a7d779d75dff96fa/users/e4b52c0d94d144e4b5bde97074ae1468/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:05.570771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 923/1884] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:05.575185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.577450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.577736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.578017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.613808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] MFA Rules not processed for user `e4b52c0d94d144e4b5bde97074ae1468`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:05.615787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:05.631937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:05.646868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334495b-eb22-4016-bddc-503c35217c31 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.651939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.652768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.653151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.653505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.654825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.656633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.666991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 962/1885] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:05.671208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.683010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f061f2-6669-4259-a3f8-9d177912e5bb admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.690195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.690872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.691157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.694729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.694873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 924/1886] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:05.694963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.694963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.703327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.704980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:05.705116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 963/1887] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:05.711910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.722077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4a2c-7959-413a-9c4a-3453f273fe53 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.727650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.728782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.729067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.729417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] PATH_INFO: `/v3/projects/cb8811e6acca469bb07fa59629cff1ca/users/4caa931a10014f408116db1ea4036ced/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.745230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8811e6acca469bb07fa59629cff1ca, user_id=4caa931a10014f408116db1ea4036ced, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.748465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846abd6-11a9-450b-ae60-73164838d1cf admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.763215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 964/1888] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:05 2022] PUT /identity/v3/projects/cb8811e6acca469bb07fa59629cff1ca/users/4caa931a10014f408116db1ea4036ced/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:05.769720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:05.774066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.774441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.774757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b246a44-9735-4879-8180-dd9c549372ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.774862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.803668 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] MFA Rules not processed for user `4caa931a10014f408116db1ea4036ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:05.804895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:05.829434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.830346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.830673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.830961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.869215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 965/1889] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:05.874156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:05.888319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.889776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:05.899296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 925/1890] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:05.944726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd417dd-876c-424c-ae6c-262c27e5cdda None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:05.949360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:05.950429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:05.950923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:05.951574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:05.952509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:05.955162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.014869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 966/1891] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:06.018403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.035158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d2a7-82cc-4e2c-88e8-7f579d02dcc8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.110145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.110841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.111122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.111400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.117489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.120022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.155522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 926/1892] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:06 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:06.159326 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.170664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa2be2-e226-4798-baee-6f6e0e78a196 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.176744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.177570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.177869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.178132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.178861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.180482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.192716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 967/1893] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:06.197220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.208850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068779a-d40b-47d8-ad6d-e738519f2b86 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.214723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.216104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.216398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.216675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] PATH_INFO: `/v3/projects/e2d8669cbc6247e6ae158ce9f7117695/users/f4c71e1fdde1476c8a631cd8d2128645/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.241300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d8669cbc6247e6ae158ce9f7117695, user_id=f4c71e1fdde1476c8a631cd8d2128645, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.244491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.266257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 927/1894] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:06 2022] PUT /identity/v3/projects/e2d8669cbc6247e6ae158ce9f7117695/users/f4c71e1fdde1476c8a631cd8d2128645/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:06.272065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.285255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d7aec-ad16-4df7-afa3-190040948330 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.301124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:06.365655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.366353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.366638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.366918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.367925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.369670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd373e-3b4a-4df6-bb9a-46e392a65a76 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.384277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 968/1895] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:06.385578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.396423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23738523-6d32-4d5b-9912-bf0a88e03d9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.402017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.402955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.403237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.403644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] PATH_INFO: `/v3/projects/e2d8669cbc6247e6ae158ce9f7117695/users/f4c71e1fdde1476c8a631cd8d2128645/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.405682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d8669cbc6247e6ae158ce9f7117695, user_id=f4c71e1fdde1476c8a631cd8d2128645, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.408359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.427867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 928/1896] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:06 2022] PUT /identity/v3/projects/e2d8669cbc6247e6ae158ce9f7117695/users/f4c71e1fdde1476c8a631cd8d2128645/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:06.437779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.438074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.438371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.464537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] MFA Rules not processed for user `f4c71e1fdde1476c8a631cd8d2128645`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:06.465724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:06.531836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 969/1897] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:06.543755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.545457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:06.556452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:06.603963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1843d-3c0d-463c-95f6-ab311164e8bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.608112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.608757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.609042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.609318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.618179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.619601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.629753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 929/1898] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:06.670176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:06.679510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ebe19-45fa-4834-ab3c-a47ba5db01cc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:06.683077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:06.684300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:06.684584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:06.684857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:06.741216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:06.742805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:06.752377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 970/1899] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:09.409504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:09.418007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431fe79-9aa5-478e-aa5f-27378a43183e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:09.421683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:09.422328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:09.422612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:09.422885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:09.424656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:09.486562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:09.488029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:09.503908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 930/1900] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:12.098140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.101755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.175552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:12.242524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4002e-ffeb-46c3-835f-d9ec4adaa779 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.256742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.257693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.257930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.258303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] PATH_INFO: `/v3/users/5fd4bb188b174b039e23582648f21fc1` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.260933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd4bb188b174b039e23582648f21fc1)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.263445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.279625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.291861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa453f31-4abf-48dd-8c14-d3cbcd619512 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.300699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.301618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.301897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.302348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] PATH_INFO: `/v3/users/a6359f6129cc42ecb571da03ee2e072f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.304004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6359f6129cc42ecb571da03ee2e072f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.305969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.343928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd4bb188b174b039e23582648f21fc1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.360593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd4bb188b174b039e23582648f21fc1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.364758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6359f6129cc42ecb571da03ee2e072f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.365573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6359f6129cc42ecb571da03ee2e072f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.370682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd4bb188b174b039e23582648f21fc1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.372026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd4bb188b174b039e23582648f21fc1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.380898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6359f6129cc42ecb571da03ee2e072f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.381210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd4bb188b174b039e23582648f21fc1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.392731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6359f6129cc42ecb571da03ee2e072f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.393833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adf36be4-3ad4-4616-9c34-141347ea72d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6359f6129cc42ecb571da03ee2e072f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.397100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 971/1901] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:12 2022] DELETE /identity/v3/users/5fd4bb188b174b039e23582648f21fc1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:12.400119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.401028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.413359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 931/1902] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:12 2022] DELETE /identity/v3/users/a6359f6129cc42ecb571da03ee2e072f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:12.414977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550cb3-95ae-4802-a7e0-2b5a8fdad975 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.415469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550cb3-95ae-4802-a7e0-2b5a8fdad975 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.415914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550cb3-95ae-4802-a7e0-2b5a8fdad975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.435197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-62550cb3-95ae-4802-a7e0-2b5a8fdad975 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:12.436690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-62550cb3-95ae-4802-a7e0-2b5a8fdad975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.459014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:12.512790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63dfb-d733-4e4a-a6e2-7e0781a8db5e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.525578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 932/1903] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:12.527006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.527296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.527570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.546825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:12.548186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.556484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 933/1904] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:12.560927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.562721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.574964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.576470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.576470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.576699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] PATH_INFO: `/v3/users/61f1caa43c1e49e1aa1dd937d52a14a1` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.579183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f1caa43c1e49e1aa1dd937d52a14a1)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.581365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.584098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:12.630034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455705cf-9f0c-4eba-ab23-7df48f7c5e3c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.638402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f1caa43c1e49e1aa1dd937d52a14a1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.641251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.641936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.642212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.642699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.652094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f1caa43c1e49e1aa1dd937d52a14a1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.653143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f1caa43c1e49e1aa1dd937d52a14a1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.653903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f1caa43c1e49e1aa1dd937d52a14a1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.655344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.657518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.661712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da4fd2d-2501-4426-8895-25c3452375d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f1caa43c1e49e1aa1dd937d52a14a1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.670975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 934/1905] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:12.671900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.673899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.679823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 972/1906] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:12 2022] DELETE /identity/v3/users/61f1caa43c1e49e1aa1dd937d52a14a1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:12.688230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.688230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.688230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.693639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfed3e-3e1a-4faf-80c5-187a99f20ec1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.708069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:12.709743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.769103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.770223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.770456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.770727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.773056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.798906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 973/1907] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:12.815633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.819040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.835114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d5dc0-a957-41ac-96a0-c0df17f428bf None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.841217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.842176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.842458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.842733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.856694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.858365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.871496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.872707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 974/1908] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:12.873209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95008c38-650f-4f3c-a013-2ad534bc0be7 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.874103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:12.875424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:12.884680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 935/1909] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:12.889461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e49b3c-127d-43f3-a6b2-a85a5dd03333 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:12.894692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:12.895571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.895945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.896226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] PATH_INFO: `/v3/users/8488e4e249f24d49a4b1c0d5a051e114` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:12.897455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8488e4e249f24d49a4b1c0d5a051e114)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:12.899373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:12.950193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8488e4e249f24d49a4b1c0d5a051e114'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.962281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:12.963517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8488e4e249f24d49a4b1c0d5a051e114'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.965244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8488e4e249f24d49a4b1c0d5a051e114'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.966430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8488e4e249f24d49a4b1c0d5a051e114'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.975645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8488e4e249f24d49a4b1c0d5a051e114'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:12.990796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 975/1910] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:12 2022] DELETE /identity/v3/users/8488e4e249f24d49a4b1c0d5a051e114 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:12.995242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a81c-4114-4de7-b980-eb65ca650bdb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:12.995639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a81c-4114-4de7-b980-eb65ca650bdb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:12.995869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a81c-4114-4de7-b980-eb65ca650bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.013405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9cf4a81c-4114-4de7-b980-eb65ca650bdb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:13.014691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9cf4a81c-4114-4de7-b980-eb65ca650bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.019127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:13.102014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 936/1911] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:13.113136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.115482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.191835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102798-7bed-479b-9d3c-0578ef740350 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.201580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:13.202437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.203797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.204247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.219887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:13.221717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c6d1-7321-473e-8be3-52380c8db7c4 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:13.248055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 976/1912] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:13.268961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.270011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.270401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.272833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 937/1913] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:37:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:37:13.288690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.289173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.289769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.297915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.301203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.358193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:13.359529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.363471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182460aa-4460-4d3b-8c5c-dd544f7c7004 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.383257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:13.387488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.388069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.389107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] PATH_INFO: `/v3/projects/5a8d856bbccf432aad87a98177312018` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.391894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8d856bbccf432aad87a98177312018)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:13.394453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:13.477960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8d856bbccf432aad87a98177312018', 'request_id': 'req-52d711a7-6fae-4fa8-9979-9268d07b2e27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.487916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8d856bbccf432aad87a98177312018', 'request_id': 'req-52d711a7-6fae-4fa8-9979-9268d07b2e27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.501993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8d856bbccf432aad87a98177312018', 'request_id': 'req-52d711a7-6fae-4fa8-9979-9268d07b2e27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.543742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] The token cache is being invalidate because project 5a8d856bbccf432aad87a98177312018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:13.547798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.552828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.554881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 938/1914] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:13 2022] DELETE /identity/v3/projects/5a8d856bbccf432aad87a98177312018 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:13.555684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.557134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.578512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 977/1915] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:13.579502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.580957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.595700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7942b-f0f7-4c16-be4e-835a7303abfe None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.695060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d711a7-6fae-4fa8-9979-9268d07b2e27 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.701490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:13.701790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:13.702379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.702661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.702771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.702925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] PATH_INFO: `/v3/projects/88ff64a7aa77424c8da614400df6bffb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.703017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.703232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] PATH_INFO: `/v3/users/018cca0fff814804bba0962b89053ec8` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.704262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ff64a7aa77424c8da614400df6bffb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:13.704498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=018cca0fff814804bba0962b89053ec8)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:13.706545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:13.707946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:13.735755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:13.765434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:13.792930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ff64a7aa77424c8da614400df6bffb', 'request_id': 'req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.798814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ff64a7aa77424c8da614400df6bffb', 'request_id': 'req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.802473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018cca0fff814804bba0962b89053ec8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.805366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ff64a7aa77424c8da614400df6bffb', 'request_id': 'req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.824542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] The token cache is being invalidate because project 88ff64a7aa77424c8da614400df6bffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:13.824979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.825742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.825857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018cca0fff814804bba0962b89053ec8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.826712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018cca0fff814804bba0962b89053ec8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.827552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018cca0fff814804bba0962b89053ec8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.828141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 939/1916] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:13 2022] DELETE /identity/v3/projects/88ff64a7aa77424c8da614400df6bffb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:13.828948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.830526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.839207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018cca0fff814804bba0962b89053ec8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:13.862917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 978/1917] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:13 2022] DELETE /identity/v3/users/018cca0fff814804bba0962b89053ec8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:13.863619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:13.866104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.936254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b9ca-6c48-4532-8424-aa4bcf41ab00 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.984677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8382b9-28a7-4392-bbf3-c1d093d3a7af admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:13.999591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.000579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.000861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.001149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] PATH_INFO: `/v3/users/4caa931a10014f408116db1ea4036ced` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.002683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4caa931a10014f408116db1ea4036ced)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.006191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.018551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.020137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.020584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.021086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] PATH_INFO: `/v3/projects/793d6eb14dd640b4bfc4b6576ae2a287` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.023468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=793d6eb14dd640b4bfc4b6576ae2a287)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.026562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.113549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4caa931a10014f408116db1ea4036ced'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.129839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793d6eb14dd640b4bfc4b6576ae2a287', 'request_id': 'req-862a2271-0f07-402d-adbb-2f265dcbd2e8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.132875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4caa931a10014f408116db1ea4036ced'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.134218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4caa931a10014f408116db1ea4036ced'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.135032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4caa931a10014f408116db1ea4036ced'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.144296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793d6eb14dd640b4bfc4b6576ae2a287', 'request_id': 'req-862a2271-0f07-402d-adbb-2f265dcbd2e8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.148218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4caa931a10014f408116db1ea4036ced'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.150633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793d6eb14dd640b4bfc4b6576ae2a287', 'request_id': 'req-862a2271-0f07-402d-adbb-2f265dcbd2e8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.174290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:14.182524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] The token cache is being invalidate because project 793d6eb14dd640b4bfc4b6576ae2a287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:14.183421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.185589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.187878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 979/1918] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:13 2022] DELETE /identity/v3/users/4caa931a10014f408116db1ea4036ced => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.189477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.190930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 940/1919] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:13 2022] DELETE /identity/v3/projects/793d6eb14dd640b4bfc4b6576ae2a287 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.198119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.200348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.202261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.234673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:14.320548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf50e-778d-41c1-a68b-ff5dc6c82766 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.322595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a2271-0f07-402d-adbb-2f265dcbd2e8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.327793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.328425 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.328967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.329252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.331798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.348723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.350544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.375979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 980/1920] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:14.377281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.379403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.410434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.411554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.412470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.412748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] PATH_INFO: `/v3/projects/a9296f4a79f348cf9a6662c2ba21dab6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.416069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9296f4a79f348cf9a6662c2ba21dab6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.426945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.472523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eb94f-73f4-4be8-bc9b-f5fbe606ef5b service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.478902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.479778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.480139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.480500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] PATH_INFO: `/v3/users/b15bf9424671466a977d9b5a1ebc6fba` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.482707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15bf9424671466a977d9b5a1ebc6fba)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.485195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.518704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9296f4a79f348cf9a6662c2ba21dab6', 'request_id': 'req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.525519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9296f4a79f348cf9a6662c2ba21dab6', 'request_id': 'req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.533950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9296f4a79f348cf9a6662c2ba21dab6', 'request_id': 'req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.548228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] The token cache is being invalidate because project a9296f4a79f348cf9a6662c2ba21dab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:14.548435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.549471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.551156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 941/1921] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:14 2022] DELETE /identity/v3/projects/a9296f4a79f348cf9a6662c2ba21dab6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.552046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.553345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.557474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15bf9424671466a977d9b5a1ebc6fba'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.572010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15bf9424671466a977d9b5a1ebc6fba'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.572850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15bf9424671466a977d9b5a1ebc6fba'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.573863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15bf9424671466a977d9b5a1ebc6fba'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.583300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15bf9424671466a977d9b5a1ebc6fba'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.598037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 981/1922] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:14 2022] DELETE /identity/v3/users/b15bf9424671466a977d9b5a1ebc6fba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.602208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:14.602396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.603745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.640413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:14.646088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e39ca-4db8-4e0d-b3bd-34511fb5a93c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.719940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea07e7d-451d-4116-a1c4-4123ecbca41c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.728366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.730667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.731045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.731342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] PATH_INFO: `/v3/users/ba4f2faf24db44f4829294a2bc6843fb` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.733952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4f2faf24db44f4829294a2bc6843fb)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.736374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.744212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.745636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.747307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.747925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] PATH_INFO: `/v3/projects/c20ba08560104d658730541ee0296273` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.749716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20ba08560104d658730541ee0296273)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.753853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.853856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4f2faf24db44f4829294a2bc6843fb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.873235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4f2faf24db44f4829294a2bc6843fb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.874867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20ba08560104d658730541ee0296273', 'request_id': 'req-9014c8ec-4638-4301-8ab7-137b869433ce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.883710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20ba08560104d658730541ee0296273', 'request_id': 'req-9014c8ec-4638-4301-8ab7-137b869433ce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.885015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4f2faf24db44f4829294a2bc6843fb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.886615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4f2faf24db44f4829294a2bc6843fb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.897820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20ba08560104d658730541ee0296273', 'request_id': 'req-9014c8ec-4638-4301-8ab7-137b869433ce'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.906093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5b4c53fc-3baa-422c-a71c-042e3f2b1f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4f2faf24db44f4829294a2bc6843fb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.926991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] The token cache is being invalidate because project c20ba08560104d658730541ee0296273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:14.927585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.928642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:14.934691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 942/1923] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:14 2022] DELETE /identity/v3/projects/c20ba08560104d658730541ee0296273 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.934848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 982/1924] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:14 2022] DELETE /identity/v3/users/ba4f2faf24db44f4829294a2bc6843fb => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:14.934937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.938221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac1d4c-8195-49e8-a52d-0a2133f81cd7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.938221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac1d4c-8195-49e8-a52d-0a2133f81cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.938221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac1d4c-8195-49e8-a52d-0a2133f81cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.941571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.963938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-beac1d4c-8195-49e8-a52d-0a2133f81cd7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:14.966880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-beac1d4c-8195-49e8-a52d-0a2133f81cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.109704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014c8ec-4638-4301-8ab7-137b869433ce admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.110061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 983/1925] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:15.113966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.113966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.114810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.119268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.121554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.121834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.122109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] PATH_INFO: `/v3/projects/cb8811e6acca469bb07fa59629cff1ca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.124104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8811e6acca469bb07fa59629cff1ca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.127416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.166177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:15.168741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.170634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:15.195087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 984/1926] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:15.196399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.208829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2f27-5ff3-431c-84df-60f2464d0d5c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.227351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8811e6acca469bb07fa59629cff1ca', 'request_id': 'req-847ed397-6d04-4912-ad72-9ef27f7968cb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.238750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8811e6acca469bb07fa59629cff1ca', 'request_id': 'req-847ed397-6d04-4912-ad72-9ef27f7968cb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.245740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:15.248542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8811e6acca469bb07fa59629cff1ca', 'request_id': 'req-847ed397-6d04-4912-ad72-9ef27f7968cb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.295209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] The token cache is being invalidate because project cb8811e6acca469bb07fa59629cff1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:15.295553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.297751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.297960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 943/1927] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:14 2022] DELETE /identity/v3/projects/cb8811e6acca469bb07fa59629cff1ca => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:15.301857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.302730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.303042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.303389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.304050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.304372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:15.304999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.307548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.307643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.324740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ed397-6d04-4912-ad72-9ef27f7968cb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.330495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.333154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.333792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.334587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 985/1928] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:15.334876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] PATH_INFO: `/v3/users/f4c71e1fdde1476c8a631cd8d2128645` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.336753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c71e1fdde1476c8a631cd8d2128645)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.341127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.343282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.343666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.368696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2dc58-d037-42e7-a131-7b0348fea898 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.374725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.375424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.375710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.376019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.376857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.380411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.426556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 986/1929] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:15 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:15.430852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.432574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.479089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c71e1fdde1476c8a631cd8d2128645'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.480564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c71e1fdde1476c8a631cd8d2128645'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.512346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c71e1fdde1476c8a631cd8d2128645'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.525178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c71e1fdde1476c8a631cd8d2128645'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.526626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c71e1fdde1476c8a631cd8d2128645'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.547179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 944/1930] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:15 2022] DELETE /identity/v3/users/f4c71e1fdde1476c8a631cd8d2128645 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:15.548660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.550549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.560411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa357232-5029-4f28-9a87-521d91c74f0f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.705172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851882f-c4f1-4b92-81e0-f0fae1289a95 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.706296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.706973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.707402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.708208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.712736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.718854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.719558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.719865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.720147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.720846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.727729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.761233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 945/1931] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:15 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:15.761233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.763235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.777935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e57e76-8e69-43fb-bea6-a171f39bde35 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.784256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:15.785190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:15.786038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:15.786323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] PATH_INFO: `/v3/projects/2b8d385f4c2d442285173b7ce74d7816` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:15.788968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8d385f4c2d442285173b7ce74d7816)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.792155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.795129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:15.862485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8d385f4c2d442285173b7ce74d7816', 'request_id': 'req-14749ca0-f4d2-44b9-9313-c05ef0a12287'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.870005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8d385f4c2d442285173b7ce74d7816', 'request_id': 'req-14749ca0-f4d2-44b9-9313-c05ef0a12287'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.880637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8d385f4c2d442285173b7ce74d7816', 'request_id': 'req-14749ca0-f4d2-44b9-9313-c05ef0a12287'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.880952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:15.887829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:15.899802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] The token cache is being invalidate because project 2b8d385f4c2d442285173b7ce74d7816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:15.900388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.901280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:15.904159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 946/1932] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:15 2022] DELETE /identity/v3/projects/2b8d385f4c2d442285173b7ce74d7816 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:15.905263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.907151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:15.913939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 987/1933] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:15.915212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:15.929957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c795-c419-4d79-b477-fb6ded50fcb4 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:15.936978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:16.018498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14749ca0-f4d2-44b9-9313-c05ef0a12287 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.026793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.028049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.028325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.028760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.030165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.032415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.045618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 947/1934] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:16.050527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.077657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.079149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.079733 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506da8c-b60a-437c-8e2e-7feaf0d792ae admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.080295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.080661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] PATH_INFO: `/v3/users/8ea6accfd5eb46fa9cff20b497a10479` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.082300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea6accfd5eb46fa9cff20b497a10479)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.086013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.095721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.096590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.097723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.097723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.098138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.099869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.112381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 948/1935] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:16.131606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.152933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded4e-fc6f-46bb-a2da-5402cf2f77ef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.160854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.162878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.163262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.163716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] PATH_INFO: `/v3/projects/5e997d08380d40c998d51ec440b05710/users/4e65c44fe1c343b1966f9d86c3b01b74/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.181225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea6accfd5eb46fa9cff20b497a10479'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.193377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e997d08380d40c998d51ec440b05710, user_id=4e65c44fe1c343b1966f9d86c3b01b74, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.198158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965085e9-8455-4b72-9597-75c82e02433c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.200643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea6accfd5eb46fa9cff20b497a10479'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.201925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea6accfd5eb46fa9cff20b497a10479'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.202687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea6accfd5eb46fa9cff20b497a10479'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.213250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea6accfd5eb46fa9cff20b497a10479'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.229123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 949/1936] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:16 2022] PUT /identity/v3/projects/5e997d08380d40c998d51ec440b05710/users/4e65c44fe1c343b1966f9d86c3b01b74/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:16.239682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.240514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.241233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.263649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 988/1937] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:15 2022] DELETE /identity/v3/users/8ea6accfd5eb46fa9cff20b497a10479 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:16.279813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] MFA Rules not processed for user `4e65c44fe1c343b1966f9d86c3b01b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:16.281140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.314218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.316209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.343025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:16.430172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 950/1938] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:16.432105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.436171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.510506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cc6e4-a3f3-4da9-821c-0199e87a133e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.518284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.520795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.521122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.521982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] PATH_INFO: `/v3/projects/e6bf50ee0542429294868c50cfd97b8a` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.522999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bf50ee0542429294868c50cfd97b8a)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.525841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.557573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3635da-0fc2-4957-a650-5d10d225971f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.563071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.563978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.564180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.564455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.565374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.567732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.573342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:16.601397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 951/1939] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:16.601397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.602698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.604039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bf50ee0542429294868c50cfd97b8a', 'request_id': 'req-11f43cfe-df37-4255-a627-e4c646a0c2d1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.613502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bf50ee0542429294868c50cfd97b8a', 'request_id': 'req-11f43cfe-df37-4255-a627-e4c646a0c2d1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.622744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bf50ee0542429294868c50cfd97b8a', 'request_id': 'req-11f43cfe-df37-4255-a627-e4c646a0c2d1'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.637810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] The token cache is being invalidate because project e6bf50ee0542429294868c50cfd97b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:16.638635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.639297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.650734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 989/1940] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:16 2022] DELETE /identity/v3/projects/e6bf50ee0542429294868c50cfd97b8a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:16.653104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.656655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.681400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab987d-e939-421f-9d2a-fa37adc6d3f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.689818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.690724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.691010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.691286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] PATH_INFO: `/v3/projects/e2d8669cbc6247e6ae158ce9f7117695` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.692692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d8669cbc6247e6ae158ce9f7117695)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.695005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.738815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f43cfe-df37-4255-a627-e4c646a0c2d1 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.748484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.749217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.749458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.749733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.750459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.752471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c72fe4-f558-4b70-8f16-4eaa1345ffd9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.775149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d8669cbc6247e6ae158ce9f7117695', 'request_id': 'req-c38ec095-aabc-4ba3-b82d-9a80162813d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.782511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d8669cbc6247e6ae158ce9f7117695', 'request_id': 'req-c38ec095-aabc-4ba3-b82d-9a80162813d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.790697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d8669cbc6247e6ae158ce9f7117695', 'request_id': 'req-c38ec095-aabc-4ba3-b82d-9a80162813d6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.791804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 990/1941] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:16.793923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6fb77-5b71-4fff-99d3-cdb64d081548 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.794218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6fb77-5b71-4fff-99d3-cdb64d081548 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.794497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6fb77-5b71-4fff-99d3-cdb64d081548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.796139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:16.810139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] The token cache is being invalidate because project e2d8669cbc6247e6ae158ce9f7117695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:16.810773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.812108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:16.813894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 952/1942] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:16 2022] DELETE /identity/v3/projects/e2d8669cbc6247e6ae158ce9f7117695 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:16.814537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.815680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-02a6fb77-5b71-4fff-99d3-cdb64d081548 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:16.815825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.817351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-02a6fb77-5b71-4fff-99d3-cdb64d081548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.894630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 991/1943] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:16.896036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.896317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.896598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.934373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec095-aabc-4ba3-b82d-9a80162813d6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.936034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:16.937230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.946693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 992/1944] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:16.947516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:16.948446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.949745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.950031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.950322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.951047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.953054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d662d0-030c-414d-8822-0fc92519c2b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.958059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:16.972454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 953/1945] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:16.972454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202950e5-5b38-4802-9ede-d085547408b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.972454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202950e5-5b38-4802-9ede-d085547408b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.972454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202950e5-5b38-4802-9ede-d085547408b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.976610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44174f-a718-4a19-8eca-1dccda193e4f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:16.981528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:16.982414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:16.982691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:16.982962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] PATH_INFO: `/v3/projects/1b4305c980174331bc51d31b6b113f97` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:16.984182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4305c980174331bc51d31b6b113f97)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:16.986170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:16.992604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-202950e5-5b38-4802-9ede-d085547408b6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:16.993925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-202950e5-5b38-4802-9ede-d085547408b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.005360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 954/1946] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:17.006589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.006982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.007364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.028971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:17.030025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:17.032462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.042528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4305c980174331bc51d31b6b113f97', 'request_id': 'req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:17.048385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4305c980174331bc51d31b6b113f97', 'request_id': 'req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:17.058079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4305c980174331bc51d31b6b113f97', 'request_id': 'req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:17.072885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] The token cache is being invalidate because project 1b4305c980174331bc51d31b6b113f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:17.073174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:17.074823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:17.076623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 993/1947] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:16 2022] DELETE /identity/v3/projects/1b4305c980174331bc51d31b6b113f97 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:17.077371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.078825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.112057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 955/1948] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:17.116334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.117678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.129461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267fde-1333-48c6-8907-47382861e7e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.134069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.134723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.135023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.135294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.136565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:17.136877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.138466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65426ec7-093e-4cc5-80a6-325f9f846a13 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.153964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 956/1949] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:17.153964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.153964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.153964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.155281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76ca49-3e69-43d6-b23e-9e7c65bbb0a3 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.159955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.160846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.161129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.161406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] PATH_INFO: `/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80/users/333234ef7f42469799f973cf906120c7/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.170171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:17.171377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.176249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2e60a92e9145428f2f3fff8ec33a80, user_id=333234ef7f42469799f973cf906120c7, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.178966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.179350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 957/1950] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:17.180108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.191344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7701e-2280-4633-af87-21684c5aa604 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.197976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 994/1951] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:17 2022] PUT /identity/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80/users/333234ef7f42469799f973cf906120c7/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:17.199390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.252605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.253275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.253560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.253838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.254592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:17.254911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.256619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187598ec-e1a8-446b-9457-2844757dea49 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.263361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00c50e-f8a9-48b9-abe8-d24798a1aecb admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.265774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:17.268279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.269043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.269398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.269747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.270371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 958/1952] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:17.270577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.271848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.272394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.272502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.272762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.294659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:17.296082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.300460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 995/1953] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.301236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.307325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 959/1954] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:17.307325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.317969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b06ec-8630-4dd1-8807-48447a38b10b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.345325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:17.390993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c51f2-8b54-4dbb-88b2-63d8fcc148a5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.394085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.395427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.395949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.396367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.397099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.397676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.398416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.398772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.399112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.399204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.399965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.401621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.417751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 996/1955] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:17.418602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.431567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3800f92-99a5-44df-be88-e7ccfbd8b54b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.433993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 960/1956] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.435704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.515314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.515314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.515314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.515314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.515635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.517363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.534251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be377e8e-7406-4135-b6fc-e2089fe9fb85 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.539886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.540803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.541231 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.541699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.542686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:17.543143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.545111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.547625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 997/1957] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.548554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.562596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2428f-3a3a-4f93-9848-0167b68f1837 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.563185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 961/1958] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:17.564217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.574165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.575892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.576560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.576937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] PATH_INFO: `/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80/users/333234ef7f42469799f973cf906120c7/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.578396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68caec-6bb5-4584-a12b-ee6293716d59 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.579333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2e60a92e9145428f2f3fff8ec33a80, user_id=333234ef7f42469799f973cf906120c7, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.582299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.584005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.585662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.586113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.586547 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.587400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.589526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.597593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 998/1959] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:17 2022] PUT /identity/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80/users/333234ef7f42469799f973cf906120c7/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:17.599158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.623364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 962/1960] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/users => generated 292 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.624454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.636403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050841e-2141-41ea-9e60-66dd196b8575 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.686977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76595f3f-9bdf-4658-9a16-a591d5e96f7e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.694331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.695060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.696796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.696796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.696796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.698567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62a920-c2e9-41ea-ab4d-fb1b9dd1484c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.711028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 999/1961] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:17.713321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.713732 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.715802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.717576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:17.726737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.727800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.728155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.728738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.729452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.732405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.743907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] MFA Rules not processed for user `333234ef7f42469799f973cf906120c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:17.745102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.770037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 963/1962] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.770037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.787256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:17.831559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1000/1963] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:17.832290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.845785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822e17-0606-444c-8286-fdc160b711d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.847242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c0f61-7ac9-4f4d-9239-46ace193c5c9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.850397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.851097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.851373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.851900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.852241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.852794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.853012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.853338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.853619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.854443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.854558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.856839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.866093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 964/1964] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:17.867393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.872323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1001/1965] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:17.872323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.872323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:17.877874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2af87-3402-4e37-a12d-be2d28414a73 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.882548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.883205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.883492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.883788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.884483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.886413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.909964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 965/1966] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:17 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:17.909964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.918497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ef-dd75-4fe2-aa97-5bfb605075fd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.922999 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.923642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.923981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.924261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.924976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.926554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:17.930713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48d4-987c-499a-8d9b-794bd2d8f78c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.934849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.935553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.935886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.936091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.939268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 966/1967] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:17.939983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:17.950511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9bf1-9329-448b-9e30-b215d7fc63a6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:17.955856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:17.956739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:17.957028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:17.957303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] PATH_INFO: `/v3/projects/26a990504016420994bab36e29f12925/users/aa5b9c4aa7c841f984a7b2f1e06866d8/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:17.978793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a990504016420994bab36e29f12925, user_id=aa5b9c4aa7c841f984a7b2f1e06866d8, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:17.981921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.001162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 967/1968] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:17 2022] PUT /identity/v3/projects/26a990504016420994bab36e29f12925/users/aa5b9c4aa7c841f984a7b2f1e06866d8/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:18.002043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.045944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.052657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.065210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1002/1969] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:18.066036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.077864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c118b13-ce23-43a9-9fb0-c8a3853b8bdb service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.084708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.085735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.086081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.086366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] PATH_INFO: `/v3/projects/c1f22b2b219c42218f2e7298f4437e76/users/9e7c75a7d4804e3c885192a33b56fa68/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.086791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735ed59-e535-4c57-87ce-ab8618c0619d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.093140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.094094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.094416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.094848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.095543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.098547 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34160e6-00a2-41a2-8e49-808e09c8f3af admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.114069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f22b2b219c42218f2e7298f4437e76, user_id=9e7c75a7d4804e3c885192a33b56fa68, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.117484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 968/1970] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:18.119619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.123550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.124298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.124298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.140662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1003/1971] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:18 2022] PUT /identity/v3/projects/c1f22b2b219c42218f2e7298f4437e76/users/9e7c75a7d4804e3c885192a33b56fa68/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:18.141434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.209681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:18.210275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] MFA Rules not processed for user `aa5b9c4aa7c841f984a7b2f1e06866d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:18.211077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:18.229752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:18.243377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f0fd8-ac13-409d-9d09-a00783761fdc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.252376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.253191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.253474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.253749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.254490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.266221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d8a23-ecc2-4239-9df9-4fab736159a3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.288825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1004/1972] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:18.290582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0d91-6690-4e2c-a82d-56f7ff127076 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.290880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0d91-6690-4e2c-a82d-56f7ff127076 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.291164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b0d91-6690-4e2c-a82d-56f7ff127076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.325932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-080b0d91-6690-4e2c-a82d-56f7ff127076 None None] MFA Rules not processed for user `9e7c75a7d4804e3c885192a33b56fa68`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:18.328099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-080b0d91-6690-4e2c-a82d-56f7ff127076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:18.337233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 969/1973] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:18.337233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.346697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8ac69-7b58-4f8d-bad1-766996b98ec0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.351294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.352215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.352742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.352742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] PATH_INFO: `/v3/projects/578711ed115145e991f4522aa8d5e84c/users/fb20ef7a924a4aafa4676fa5e094d3c5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.368843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=578711ed115145e991f4522aa8d5e84c, user_id=fb20ef7a924a4aafa4676fa5e094d3c5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.372389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.392070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 970/1974] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:18 2022] PUT /identity/v3/projects/578711ed115145e991f4522aa8d5e84c/users/fb20ef7a924a4aafa4676fa5e094d3c5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:18.392070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.420107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1005/1975] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:18.423649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.424113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.424585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.463262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] MFA Rules not processed for user `fb20ef7a924a4aafa4676fa5e094d3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:18.464787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:18.489321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1132f3f-5440-494a-95bb-60fa303d9b7e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.494252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.494919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.495259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.495598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.497074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.499054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.531490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 971/1976] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.531490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.544656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1006/1977] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:18.544656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.562558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14a35d-1929-4d26-92a7-81ca0523203f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.598580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabc0ce-be70-48ca-a6a7-5d47f315bb9f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.603176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.603877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.604164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.604445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.605796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.607706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.611753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:18.618090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.618756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.619036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.619311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.620084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.622852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.624614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:18.638153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 972/1978] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.639051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.646554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1007/1979] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.647365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.701707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33462b3-3c0d-4dac-92aa-d4e1ee7deff1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.706869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.707535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.707849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.708128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.708831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.709297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb6c73-17ec-4755-b8b5-d4f75fd67bb3 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.710730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.741244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 973/1980] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.741244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.768066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333234ef7f42469799f973cf906120c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2e60a92e9145428f2f3fff8ec33a80', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.768740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.769026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.769304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.770040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:18.770357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.771979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.784196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1008/1981] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:18.784196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.795652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d4822-b038-4ce4-976d-8a844ea47da4 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.807636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0e91d-7f19-4716-87fd-50bdd1ea6ddf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.812571 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.813251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.813532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.813858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.814558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.816512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.850241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 974/1982] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.850241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.859180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518f06-0582-4d1a-9d04-5da9113e87bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.864720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.865225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.865457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.865750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.865897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.866032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.866179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.866455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.866708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.867207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.868800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.868900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.882880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1009/1983] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:18.884000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.896673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 975/1984] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/users => generated 299 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:18.896673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.907329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7122bf-d851-4314-b453-26a57ab18966 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.913079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.914122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.914471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.914823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.915638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.917955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.921994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:18.933277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 976/1985] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:18.934104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:18.946136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8996ef2-ca8d-415f-ba06-cf559875d47b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.951538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.952678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.953029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.953469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] PATH_INFO: `/v3/projects/84cfe2cd311c45678142b636545c831b/users/4912fcdc26ab4de8a13202900a9b0141/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.970086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cfe2cd311c45678142b636545c831b, user_id=4912fcdc26ab4de8a13202900a9b0141, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.970414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47b710-22da-4a2c-99d4-f8fc8bae74b3 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:18.972851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.974917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333234ef7f42469799f973cf906120c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2e60a92e9145428f2f3fff8ec33a80', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:18.975574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:18.975888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:18.976170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:18.976865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:18.978889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:18.988959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 977/1986] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:18 2022] PUT /identity/v3/projects/84cfe2cd311c45678142b636545c831b/users/4912fcdc26ab4de8a13202900a9b0141/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:18.991531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.013159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:19.014457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1010/1987] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:18 2022] POST /identity/v3/projects => generated 354 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:19.015571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.057669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855a15b-a5a3-4d99-877b-0df3940fbe76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.079346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ff84d-744f-4609-82ca-1a559c9b535d tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.084160 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.085275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.085778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.085890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] PATH_INFO: `/v3/projects/6c51c4dd308c49678d5e25b119d5f582/users/300db32004064bb283844ff8bbef364c/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.101568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c51c4dd308c49678d5e25b119d5f582, user_id=300db32004064bb283844ff8bbef364c, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.104633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.115938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.116749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.117036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.117460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1011/1988] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:19 2022] PUT /identity/v3/projects/6c51c4dd308c49678d5e25b119d5f582/users/300db32004064bb283844ff8bbef364c/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.117671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.118637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.118745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.120032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.134115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 978/1989] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:19.134115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.181694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037c0bc-ddde-402d-8174-63609154f766 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.186600 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.187268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.187555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.187869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.188605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.190205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.194967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529e9a-148d-467f-a7a1-97ba749b9777 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.201362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1012/1990] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:19.202413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.213484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f103-3b7b-4736-90c6-262e98ee7e02 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.219117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.220212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.220508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.220790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.221658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.223998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.240030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1013/1991] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:19.240030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.251370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece09db8-fbe0-4c5f-b938-848670984a0f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.260535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.262170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.262485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.262764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] PATH_INFO: `/v3/projects/69193596003b40a7bd26ddffdcbc8aa5/users/2bfdc8df4cd3426c928f807491b7f68d/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.263844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333234ef7f42469799f973cf906120c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2e60a92e9145428f2f3fff8ec33a80', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.264528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.264813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.265097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.265838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:19.266158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.267791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.282018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69193596003b40a7bd26ddffdcbc8aa5, user_id=2bfdc8df4cd3426c928f807491b7f68d, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.285726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 979/1992] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:19.285726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.286741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.289500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:19.305785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1014/1993] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:19 2022] PUT /identity/v3/projects/69193596003b40a7bd26ddffdcbc8aa5/users/2bfdc8df4cd3426c928f807491b7f68d/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.306906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.309763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:19.324874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56c775-3331-46e8-a1f3-e8b7c0665562 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.391538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029263b5-fa4c-4429-a896-6942ed5a9b06 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.396926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.397900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.398190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.398302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.398493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] PATH_INFO: `/v3/projects/84cfe2cd311c45678142b636545c831b/users/4912fcdc26ab4de8a13202900a9b0141/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.399229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.399619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.400059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] PATH_INFO: `/v3/projects/6c51c4dd308c49678d5e25b119d5f582/users/300db32004064bb283844ff8bbef364c/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.400606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cfe2cd311c45678142b636545c831b, user_id=4912fcdc26ab4de8a13202900a9b0141, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.402788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c51c4dd308c49678d5e25b119d5f582, user_id=300db32004064bb283844ff8bbef364c, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.403919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.406118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.416267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 980/1994] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:19 2022] PUT /identity/v3/projects/84cfe2cd311c45678142b636545c831b/users/4912fcdc26ab4de8a13202900a9b0141/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.416790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.429640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1015/1995] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:19 2022] PUT /identity/v3/projects/6c51c4dd308c49678d5e25b119d5f582/users/300db32004064bb283844ff8bbef364c/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.430665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.517793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e85f5-105f-42e7-8710-db8700723af0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.523440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e91d6-d1ad-4613-8b14-43cc7e743eaa admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.529524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333234ef7f42469799f973cf906120c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2e60a92e9145428f2f3fff8ec33a80', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.530421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.530710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.530992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] PATH_INFO: `/v3/projects/06073667bcd74479922800c24cf29915` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.532257 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06073667bcd74479922800c24cf29915)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.534186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.590574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.591273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.591556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.591911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.592650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.594499 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35007bb-772d-439f-aaad-0463ed2eca19 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.615822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 981/1996] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:19.615822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.615822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.615822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.630400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06073667bcd74479922800c24cf29915', 'request_id': 'req-8803787b-8651-450f-8619-2178b67d28b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:19.640550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06073667bcd74479922800c24cf29915', 'request_id': 'req-8803787b-8651-450f-8619-2178b67d28b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:19.650191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] MFA Rules not processed for user `4912fcdc26ab4de8a13202900a9b0141`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:19.651191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06073667bcd74479922800c24cf29915', 'request_id': 'req-8803787b-8651-450f-8619-2178b67d28b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:19.651440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:19.668404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] The token cache is being invalidate because project 06073667bcd74479922800c24cf29915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:19.669272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:19.669466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8803787b-8651-450f-8619-2178b67d28b2 tempest-VolumeQuotaClassesTest-74493607 tempest-VolumeQuotaClassesTest-74493607-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:19.672888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1016/1997] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:19 2022] DELETE /identity/v3/projects/06073667bcd74479922800c24cf29915 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.677198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.677494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.678235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.707853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] MFA Rules not processed for user `300db32004064bb283844ff8bbef364c`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:19.709440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:19.712430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:19.716458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:19.773873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 982/1998] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:19.774580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.777255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:19.840503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1017/1999] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:19.841714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:19.847360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:19.865592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a84e7-4927-493b-9c99-3860cbbc2706 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.870509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.871815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.872175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.872524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] PATH_INFO: `/v3/projects/69193596003b40a7bd26ddffdcbc8aa5/users/2bfdc8df4cd3426c928f807491b7f68d/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.874679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69193596003b40a7bd26ddffdcbc8aa5, user_id=2bfdc8df4cd3426c928f807491b7f68d, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:19.877471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12226def-c4ca-40aa-9d9c-a7d32c4bb3e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:19.889721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 983/2000] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:19 2022] PUT /identity/v3/projects/69193596003b40a7bd26ddffdcbc8aa5/users/2bfdc8df4cd3426c928f807491b7f68d/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:19.902480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.903023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.903409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.914529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97d2f7-253b-486d-a4df-5d49a62a998b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:19.929304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] MFA Rules not processed for user `2bfdc8df4cd3426c928f807491b7f68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:19.930508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:19.962662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:19.963316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:19.963603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:19.963986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:19.999702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 984/2001] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:20.003688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.003688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.013904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882792e-2dfd-4b81-9b58-52f9116df4ab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.018424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.019263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.019558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.019863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.026419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.028503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.038423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1018/2002] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:20.063572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:20.087471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.089100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.091285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.092801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.100460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 985/2003] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:20.105729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:20.114779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.125636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77599ff-3afa-4adf-9dfc-f8d45629a266 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.131147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.132415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.132934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.133430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.147942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.149375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.161196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 986/2004] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:20.186123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cca80-603d-47f4-a72e-c539a35e1736 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.189855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.190561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.190844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.191119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] PATH_INFO: `/v3/users/4e65c44fe1c343b1966f9d86c3b01b74` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.193763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e65c44fe1c343b1966f9d86c3b01b74)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.195894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.241208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.253663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5b6b6-3ce6-4f38-91ed-4f58d70079f8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.255124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.255803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.256206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.256542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.258385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.266336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e65c44fe1c343b1966f9d86c3b01b74'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.279336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e65c44fe1c343b1966f9d86c3b01b74'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.280249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e65c44fe1c343b1966f9d86c3b01b74'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.281065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e65c44fe1c343b1966f9d86c3b01b74'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.292893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8af6bd7f-1349-42f4-95ed-dec52edb99d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e65c44fe1c343b1966f9d86c3b01b74'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.310435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1019/2005] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:20 2022] DELETE /identity/v3/users/4e65c44fe1c343b1966f9d86c3b01b74 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:20.314457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.314859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.315144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.329427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.330888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.333420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:20.334615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.343889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 987/2006] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:20.345874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.357484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98424-65fa-4a4d-bd11-7775e405ac8a service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.408134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1020/2007] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:20.418738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.420964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.451875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.452562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.452846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.453122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.458298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.470663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:20.526226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f602a21-2d84-4f17-8fc0-b903715f70fb None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.535542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.536451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.536821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.537909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.552568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.554397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.581970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1021/2008] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:20.583507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.584063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:20.585160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e92e-75bf-490f-aa70-e96c58ffad16 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.587205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.610820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 988/2009] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:20.629205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19367c3d-0bf4-4936-930f-3ecb4a5d0428 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:20.642730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:20.646185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.646508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.646785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] PATH_INFO: `/v3/users/0b412e35f2f34ec89176a48da1bb85b8` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.650792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b412e35f2f34ec89176a48da1bb85b8)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:20.658940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:20.760771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b412e35f2f34ec89176a48da1bb85b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.793399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b412e35f2f34ec89176a48da1bb85b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.794209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b412e35f2f34ec89176a48da1bb85b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.796980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b412e35f2f34ec89176a48da1bb85b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.813242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b412e35f2f34ec89176a48da1bb85b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:20.855277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1022/2010] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:20 2022] DELETE /identity/v3/users/0b412e35f2f34ec89176a48da1bb85b8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:20.857198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:20.857531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:20.857819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:20.895007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:20.899382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:20.935450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:21.002797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.004269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.068394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 989/2011] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:21.079251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.084791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.140713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b554-ea07-47a7-aac4-167c2e913d2d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.150233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.151124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.151413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.152290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] PATH_INFO: `/v3/projects/5e997d08380d40c998d51ec440b05710` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.153520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e997d08380d40c998d51ec440b05710)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.155453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.168990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f4ac-6ce4-451a-bd36-f1962298030e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.183193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.184483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.184935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.185358 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.204072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.205737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.214998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:21.218772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e997d08380d40c998d51ec440b05710', 'request_id': 'req-cfa6c4fe-c145-41e0-81f0-1f04e602e985'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.220097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 990/2012] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:21.221271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.223366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.226567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e997d08380d40c998d51ec440b05710', 'request_id': 'req-cfa6c4fe-c145-41e0-81f0-1f04e602e985'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.240068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e997d08380d40c998d51ec440b05710', 'request_id': 'req-cfa6c4fe-c145-41e0-81f0-1f04e602e985'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.240540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72164aa-7c55-4e55-858f-7d86aad2671b service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.268867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] The token cache is being invalidate because project 5e997d08380d40c998d51ec440b05710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:21.269375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.269851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.271522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1023/2013] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:21 2022] DELETE /identity/v3/projects/5e997d08380d40c998d51ec440b05710 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:21.275155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.277270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.321504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.322469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.322714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.322993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] PATH_INFO: `/v3/users/9e7c75a7d4804e3c885192a33b56fa68` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.324786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7c75a7d4804e3c885192a33b56fa68)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.326930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.361209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6c4fe-c145-41e0-81f0-1f04e602e985 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.366963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.367492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.367801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.368085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] PATH_INFO: `/v3/users/fb20ef7a924a4aafa4676fa5e094d3c5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.369308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb20ef7a924a4aafa4676fa5e094d3c5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.371536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.384603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7c75a7d4804e3c885192a33b56fa68'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.385500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7c75a7d4804e3c885192a33b56fa68'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.400196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:21.402810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7c75a7d4804e3c885192a33b56fa68'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.418797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7c75a7d4804e3c885192a33b56fa68'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.420064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7c75a7d4804e3c885192a33b56fa68'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.439580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb20ef7a924a4aafa4676fa5e094d3c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.446962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 991/2014] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:21 2022] DELETE /identity/v3/users/9e7c75a7d4804e3c885192a33b56fa68 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:21.446962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.447540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.457961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb20ef7a924a4aafa4676fa5e094d3c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.458792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb20ef7a924a4aafa4676fa5e094d3c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.459484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb20ef7a924a4aafa4676fa5e094d3c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.469666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-453dc26a-e806-4ac2-b77d-6082663230dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb20ef7a924a4aafa4676fa5e094d3c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.483820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1024/2015] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:21 2022] DELETE /identity/v3/users/fb20ef7a924a4aafa4676fa5e094d3c5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:21.485268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb18d0-5a76-4e2a-9be9-328afc9d85f2 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.485564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb18d0-5a76-4e2a-9be9-328afc9d85f2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.485838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb18d0-5a76-4e2a-9be9-328afc9d85f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.507860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3dcb18d0-5a76-4e2a-9be9-328afc9d85f2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:21.509290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3dcb18d0-5a76-4e2a-9be9-328afc9d85f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.523288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953856-e091-4d09-b700-2ee0d788d6fb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.574114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:21.597457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1025/2016] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:21.598276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.598276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.598276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.612136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.613078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.613449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.613792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] PATH_INFO: `/v3/users/333234ef7f42469799f973cf906120c7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.616032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=333234ef7f42469799f973cf906120c7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.618520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.631297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:21.632785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.645455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1026/2017] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:21.646590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.648219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.663360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee5cad-31ab-4353-ba42-76c0976821c8 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.671740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.672648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.672994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.673207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] PATH_INFO: `/v3/projects/1f5e5c49b19540a085ecc519b672cb7e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.674447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5e5c49b19540a085ecc519b672cb7e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:21.676442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:21.691485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333234ef7f42469799f973cf906120c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.692691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333234ef7f42469799f973cf906120c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.719348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333234ef7f42469799f973cf906120c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.728386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333234ef7f42469799f973cf906120c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.729193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333234ef7f42469799f973cf906120c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.750275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 992/2018] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:21 2022] DELETE /identity/v3/users/333234ef7f42469799f973cf906120c7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:21.751392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.759423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5e5c49b19540a085ecc519b672cb7e', 'request_id': 'req-e718b8fb-029d-4a95-80e1-9a15a04b0c52'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.770057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.770468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5e5c49b19540a085ecc519b672cb7e', 'request_id': 'req-e718b8fb-029d-4a95-80e1-9a15a04b0c52'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.784525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5e5c49b19540a085ecc519b672cb7e', 'request_id': 'req-e718b8fb-029d-4a95-80e1-9a15a04b0c52'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.796893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:21.804026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] The token cache is being invalidate because project 1f5e5c49b19540a085ecc519b672cb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:21.804572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.805406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:21.807317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1027/2019] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:21 2022] DELETE /identity/v3/projects/1f5e5c49b19540a085ecc519b672cb7e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:21.811652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:21.813179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.900463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56539aa-c0ad-40bb-82b5-7ac6f4dd7f9e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.908515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.909310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.909661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.909839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718b8fb-029d-4a95-80e1-9a15a04b0c52 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:21.910206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.912265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:21.915455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:21.916549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:21.916958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:21.917394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:21.920260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.017919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:22.020800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:22.040825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 993/2020] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:22.042736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:22.046121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.061004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:22.062870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:22.066376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd07-e660-4b2a-9a59-5895d4439037 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:22.071077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:22.071996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:22.072282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:22.072560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] PATH_INFO: `/v3/users/e4b52c0d94d144e4b5bde97074ae1468` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:22.074718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b52c0d94d144e4b5bde97074ae1468)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:22.076868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:22.088491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1028/2021] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:22.150938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b52c0d94d144e4b5bde97074ae1468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.153269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b52c0d94d144e4b5bde97074ae1468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.178056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b52c0d94d144e4b5bde97074ae1468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.197919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b52c0d94d144e4b5bde97074ae1468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.198958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-38384b4f-007d-4f76-97d9-151c7d599203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b52c0d94d144e4b5bde97074ae1468'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.216495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 994/2022] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:22 2022] DELETE /identity/v3/users/e4b52c0d94d144e4b5bde97074ae1468 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:22.534949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:22.536543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.549637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b9ff-021b-4d81-abb1-9dddfda23b6f service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:22.570444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:22.634163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:22.635519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:22.635837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:22.636194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] PATH_INFO: `/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:22.637442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2e60a92e9145428f2f3fff8ec33a80)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:22.639411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:22.696623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2e60a92e9145428f2f3fff8ec33a80', 'request_id': 'req-80383b50-77ad-4b75-916d-86a9da8ec2b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.703531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2e60a92e9145428f2f3fff8ec33a80', 'request_id': 'req-80383b50-77ad-4b75-916d-86a9da8ec2b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.718079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2e60a92e9145428f2f3fff8ec33a80', 'request_id': 'req-80383b50-77ad-4b75-916d-86a9da8ec2b8'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.745698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] The token cache is being invalidate because project ef2e60a92e9145428f2f3fff8ec33a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:22.747060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.748545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:22.750466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1029/2023] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:22 2022] DELETE /identity/v3/projects/ef2e60a92e9145428f2f3fff8ec33a80 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:22.755301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6907-3202-43ff-995e-9be3e956fe77 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:22.757335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6907-3202-43ff-995e-9be3e956fe77 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:22.757540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e6907-3202-43ff-995e-9be3e956fe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:22.781131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac8e6907-3202-43ff-995e-9be3e956fe77 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:22.783316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac8e6907-3202-43ff-995e-9be3e956fe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.819908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:22.866608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:22.868564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.910720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 995/2024] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:22.918503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:22.918872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:22.919197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:22.943494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:22.944872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.966752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 996/2025] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:22.967857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:22.977636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383b50-77ad-4b75-916d-86a9da8ec2b8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:22.983362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:22.983873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:22.985021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:22.985402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:22.985774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] PATH_INFO: `/v3/projects/7efc76c2b13340f0a7d779d75dff96fa` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:22.987217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efc76c2b13340f0a7d779d75dff96fa)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:22.989580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.001008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5f08-1c6e-4c9b-a0cf-12e87be3224b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.010188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.012012 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.012588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.013028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] PATH_INFO: `/v3/projects/c1f22b2b219c42218f2e7298f4437e76` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.014739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f22b2b219c42218f2e7298f4437e76)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.017235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.081595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efc76c2b13340f0a7d779d75dff96fa', 'request_id': 'req-11327af7-a482-4fa3-bb09-fd576fa9f7af'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.083987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:23.090947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efc76c2b13340f0a7d779d75dff96fa', 'request_id': 'req-11327af7-a482-4fa3-bb09-fd576fa9f7af'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.096052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f22b2b219c42218f2e7298f4437e76', 'request_id': 'req-0f49a2a7-0a62-4986-8041-071628185cbd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.103199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efc76c2b13340f0a7d779d75dff96fa', 'request_id': 'req-11327af7-a482-4fa3-bb09-fd576fa9f7af'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.107100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f22b2b219c42218f2e7298f4437e76', 'request_id': 'req-0f49a2a7-0a62-4986-8041-071628185cbd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.118244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f22b2b219c42218f2e7298f4437e76', 'request_id': 'req-0f49a2a7-0a62-4986-8041-071628185cbd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.119487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] The token cache is being invalidate because project 7efc76c2b13340f0a7d779d75dff96fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:23.120038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.121331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.128078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1030/2026] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:22 2022] DELETE /identity/v3/projects/7efc76c2b13340f0a7d779d75dff96fa => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.128078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.128078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.138600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] The token cache is being invalidate because project c1f22b2b219c42218f2e7298f4437e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:23.139175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.140549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.143478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 997/2027] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:22 2022] DELETE /identity/v3/projects/c1f22b2b219c42218f2e7298f4437e76 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.144634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.147624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.237464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11327af7-a482-4fa3-bb09-fd576fa9f7af admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.240975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a2a7-0a62-4986-8041-071628185cbd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.243250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.244157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.244437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.244708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] PATH_INFO: `/v3/projects/578711ed115145e991f4522aa8d5e84c` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.247486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.248338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578711ed115145e991f4522aa8d5e84c)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.248871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.249279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.249657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.250496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:23.250610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.250982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.253345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4e2cd-de5a-4342-b9d5-0e7cc8033445 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.270928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 998/2028] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:23.277278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.277682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.278061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.300639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:23.302010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.312909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 999/2029] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:23.313283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.320406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.331206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578711ed115145e991f4522aa8d5e84c', 'request_id': 'req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.344570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578711ed115145e991f4522aa8d5e84c', 'request_id': 'req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.349182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f9fee-a375-4ea8-8405-850d1d25ebdd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.354280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578711ed115145e991f4522aa8d5e84c', 'request_id': 'req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.370944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] The token cache is being invalidate because project 578711ed115145e991f4522aa8d5e84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:23.371725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.372780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.378359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1031/2030] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:23 2022] DELETE /identity/v3/projects/578711ed115145e991f4522aa8d5e84c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.379289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.380944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.399197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:23.447184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.448121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.449356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.449647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] PATH_INFO: `/v3/users/300db32004064bb283844ff8bbef364c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.450914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=300db32004064bb283844ff8bbef364c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.452987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.472393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd94e-cb6b-4474-8e38-bd278ceb9ee6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.478734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.479473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.480443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.480791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.481587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.483882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac4926-a4be-425e-9dce-6dd6195fbf10 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.500353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:23.508781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300db32004064bb283844ff8bbef364c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.509727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300db32004064bb283844ff8bbef364c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.518680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1032/2031] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:23 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:23.520176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.520470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.520750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.527730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300db32004064bb283844ff8bbef364c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.536244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300db32004064bb283844ff8bbef364c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.536956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300db32004064bb283844ff8bbef364c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.549352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:23.550933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.556403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1000/2032] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:23 2022] DELETE /identity/v3/users/300db32004064bb283844ff8bbef364c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.557303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.558722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.623641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1033/2033] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:23.624725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.627539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.645623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585bc2f-f33e-4c68-b400-9e710a41a5fc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.725393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8779f-d085-4d07-bb4a-be12cb7d1452 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:23.735095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.736304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.736717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.737083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] PATH_INFO: `/v3/users/2bfdc8df4cd3426c928f807491b7f68d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.738720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bfdc8df4cd3426c928f807491b7f68d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.740383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:23.740928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.741625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:23.741915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:23.742260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] PATH_INFO: `/v3/users/aa5b9c4aa7c841f984a7b2f1e06866d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:23.744620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5b9c4aa7c841f984a7b2f1e06866d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:23.749191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:23.832409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bfdc8df4cd3426c928f807491b7f68d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.847051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5b9c4aa7c841f984a7b2f1e06866d8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.848739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5b9c4aa7c841f984a7b2f1e06866d8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.859123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bfdc8df4cd3426c928f807491b7f68d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.860031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bfdc8df4cd3426c928f807491b7f68d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.860537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:23.860656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bfdc8df4cd3426c928f807491b7f68d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.878494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bfdc8df4cd3426c928f807491b7f68d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.879543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5b9c4aa7c841f984a7b2f1e06866d8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.896279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5b9c4aa7c841f984a7b2f1e06866d8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.897443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5b9c4aa7c841f984a7b2f1e06866d8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:23.910531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1034/2034] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:23 2022] DELETE /identity/v3/users/2bfdc8df4cd3426c928f807491b7f68d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.911557 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.913532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.925308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1001/2035] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:23 2022] DELETE /identity/v3/users/aa5b9c4aa7c841f984a7b2f1e06866d8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:23.926075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:23.929329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:23.948952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:24.019678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dff613-0f4f-4aba-84d6-b36b1ed31c52 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.046365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbfb2-cce5-4094-a13c-35b376712630 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.056492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.057414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.058330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.059279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.060734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:24.060859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.064015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97bdc2-9fd4-49d1-a057-ba8e61c04194 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.082528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1002/2036] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:24.084092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.084488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.085404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.132948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:24.133853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.134379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.134908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.135404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.136566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.137964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.142210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.145985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1003/2037] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:24.147100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.151844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.165918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90b728-8792-45e7-b6c4-c83433cf875b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.173850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.175067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.175262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1035/2038] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:23 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:24.179814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.179814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.180708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.182642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.185522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.188077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.210399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f9ae-ee7a-4aea-9d4b-ab311863a959 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.226679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.227822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.228484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1004/2039] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:24.228616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.229084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] PATH_INFO: `/v3/projects/6c51c4dd308c49678d5e25b119d5f582` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.230078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.230795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c51c4dd308c49678d5e25b119d5f582)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.238650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.244473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.332137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c51c4dd308c49678d5e25b119d5f582', 'request_id': 'req-74681060-13ec-4081-9c60-22e5ee7461e6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.345789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c51c4dd308c49678d5e25b119d5f582', 'request_id': 'req-74681060-13ec-4081-9c60-22e5ee7461e6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.373005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c51c4dd308c49678d5e25b119d5f582', 'request_id': 'req-74681060-13ec-4081-9c60-22e5ee7461e6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.395909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] The token cache is being invalidate because project 6c51c4dd308c49678d5e25b119d5f582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:24.395909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.395909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.399823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1036/2040] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:24 2022] DELETE /identity/v3/projects/6c51c4dd308c49678d5e25b119d5f582 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:24.399823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.399823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.430396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074cf7-a828-4f08-bb48-56f077b10947 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.469328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:24.495909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.496843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.497121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.497398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.499326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.500785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74681060-13ec-4081-9c60-22e5ee7461e6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.505452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.506117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.506403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.506678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.507408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.509028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.509993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:24.512810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.514689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.520859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1037/2041] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:24.521832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.523971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.531159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1005/2042] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:24.532019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.533504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.537450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9678e5-944e-4189-9443-c341176a3eb5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.543843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.544836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.545295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.545714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.546556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.546671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266b43e-0d87-4d00-910f-1c69f8b7bcfe service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.548746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0b00d-497a-47f8-b677-9d0d7cec8048 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.554187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.555872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.556373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.556853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] PATH_INFO: `/v3/projects/69193596003b40a7bd26ddffdcbc8aa5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.559212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] RBAC: Authorizing `identity:delete_project(project_id=69193596003b40a7bd26ddffdcbc8aa5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.561542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.587492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1038/2043] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/users => generated 291 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:24.587492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.587492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.587492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.628311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:24.630422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.641672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69193596003b40a7bd26ddffdcbc8aa5', 'request_id': 'req-1066674d-1e1e-40e9-8514-5fd60749502a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.651289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69193596003b40a7bd26ddffdcbc8aa5', 'request_id': 'req-1066674d-1e1e-40e9-8514-5fd60749502a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.665769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69193596003b40a7bd26ddffdcbc8aa5', 'request_id': 'req-1066674d-1e1e-40e9-8514-5fd60749502a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.689112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] The token cache is being invalidate because project 69193596003b40a7bd26ddffdcbc8aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:24.689556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.690372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:24.692968 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1006/2044] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:24 2022] DELETE /identity/v3/projects/69193596003b40a7bd26ddffdcbc8aa5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:24.694026 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.697596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.711607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1039/2045] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:24.712347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.713692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.742299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5b29-e2a6-438d-9ef3-2d66f4ff986a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.751683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.752674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.753112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.753539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.754427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.756689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1326b9-ff87-432a-80b4-dfeb106a7c4e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.772091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1040/2046] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:24.773762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49120ae-40f5-40de-9c16-0f438c5c7c95 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.774280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49120ae-40f5-40de-9c16-0f438c5c7c95 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.774699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49120ae-40f5-40de-9c16-0f438c5c7c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.796616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066674d-1e1e-40e9-8514-5fd60749502a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.797267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f49120ae-40f5-40de-9c16-0f438c5c7c95 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:24.798477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f49120ae-40f5-40de-9c16-0f438c5c7c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.802594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.803506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.804020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.804831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.805714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.808924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.815112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1041/2047] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:24.816614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.816914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.817198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.823950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1007/2048] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:24.825258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.840326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28801-7ac9-4626-a27b-e0530d982607 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.845464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:24.846455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.846777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.849073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.849519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.849892 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.851042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.853350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da27ee-b72d-4572-b16a-e250bbb4b9b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.868414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1008/2049] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:24.868414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937840-5b47-4130-86e1-870af409a70c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.868414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937840-5b47-4130-86e1-870af409a70c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.868414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d937840-5b47-4130-86e1-870af409a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.868868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1042/2050] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:24.872083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.894138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71359-2855-4fc0-82db-922febd86b26 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.896233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d937840-5b47-4130-86e1-870af409a70c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:24.897935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d937840-5b47-4130-86e1-870af409a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.900533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.902043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.902602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.903309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] PATH_INFO: `/v3/projects/dff5e95f55664edfb76fdfe6e53eea40/users/4a55c4d6e33c4fa59f1107166e10b8c9/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.921278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1009/2051] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:24.922202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937840-5b47-4130-86e1-870af409a70c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.929755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff5e95f55664edfb76fdfe6e53eea40, user_id=4a55c4d6e33c4fa59f1107166e10b8c9, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.933247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.934027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d937840-5b47-4130-86e1-870af409a70c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.939609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.940714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.941724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.942113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] PATH_INFO: `/v3/projects/68125869b066435cb99a781bc3eba8f0/users/3c0c279fe9ef4148be77d332aae6bafb/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.959113 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=68125869b066435cb99a781bc3eba8f0, user_id=3c0c279fe9ef4148be77d332aae6bafb, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.960848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1043/2052] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:24 2022] PUT /identity/v3/projects/dff5e95f55664edfb76fdfe6e53eea40/users/4a55c4d6e33c4fa59f1107166e10b8c9/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:24.962001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.971412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.978464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1010/2053] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:24 2022] PUT /identity/v3/projects/68125869b066435cb99a781bc3eba8f0/users/3c0c279fe9ef4148be77d332aae6bafb/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:24.979671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.986568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.029340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.041900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bf67d-ae29-4219-8882-1ceeedfcac83 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.052306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.072140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6240858-cb65-4f04-b87a-19cdee1062bb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.081117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.081997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.082286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.082566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] PATH_INFO: `/v3/projects/26a990504016420994bab36e29f12925` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.083938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a990504016420994bab36e29f12925)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.086414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.103880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.105032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.105032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.105250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.105829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:25.106132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.107794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.121258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1044/2054] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:25.122185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.133130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac5b6c-ef94-4d41-b8a0-75690895ab0f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.146421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a990504016420994bab36e29f12925', 'request_id': 'req-9d7e9811-14b4-4ce2-9cec-969d30d7b705'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.153224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a990504016420994bab36e29f12925', 'request_id': 'req-9d7e9811-14b4-4ce2-9cec-969d30d7b705'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.160275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a990504016420994bab36e29f12925', 'request_id': 'req-9d7e9811-14b4-4ce2-9cec-969d30d7b705'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.174526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] The token cache is being invalidate because project 26a990504016420994bab36e29f12925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:25.174994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.175746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9d7e9811-14b4-4ce2-9cec-969d30d7b705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.178010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1011/2055] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:24 2022] DELETE /identity/v3/projects/26a990504016420994bab36e29f12925 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:25.182133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.182553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.183043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.195240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.195960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.196246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.196527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.197262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:25.197587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.200130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af58fc-0bb9-4856-a6c6-67de2164f163 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.217468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1045/2056] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:25.219116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.219415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.219697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.220545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] MFA Rules not processed for user `4a55c4d6e33c4fa59f1107166e10b8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:25.222233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.249743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] MFA Rules not processed for user `3c0c279fe9ef4148be77d332aae6bafb`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:25.251083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.305779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1012/2057] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:25.306672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.308174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.323210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b51234-8b83-4dc7-8981-946f190e3b2d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.328698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.329366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.329652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.329932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.330661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.332606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.346990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1046/2058] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:25.347892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.349904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.365018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac265c-4805-4b05-b604-f47677622219 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.365827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1013/2059] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:25.367019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.368940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.402306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.413851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.459785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.461312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.461735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.462242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] PATH_INFO: `/v3/users/4912fcdc26ab4de8a13202900a9b0141` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.463698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4912fcdc26ab4de8a13202900a9b0141)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.466518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.481653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeb1dc-9e63-4dde-be66-0fee11182d62 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.493123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.494067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.496436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.496796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.498961 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.501066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.544476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1014/2060] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:25.545458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.547161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.554117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4912fcdc26ab4de8a13202900a9b0141'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.569823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4912fcdc26ab4de8a13202900a9b0141'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.570554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4912fcdc26ab4de8a13202900a9b0141'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.571182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4912fcdc26ab4de8a13202900a9b0141'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.580825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4912fcdc26ab4de8a13202900a9b0141'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:25.604723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1047/2061] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:25 2022] DELETE /identity/v3/users/4912fcdc26ab4de8a13202900a9b0141 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:25.606189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.607726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.647678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606130c5-e6a9-42f9-9ac4-ad92f86b656f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.692758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cbdc3-2791-4765-8a77-591a89146e11 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.699901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.701679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.701973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.702316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.704498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.751754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.752548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.752910 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.753261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.754059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.756423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.782347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.783937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.792797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1015/2062] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/projects => generated 387 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:25.793814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.794748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1048/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:25.795836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.796073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.797142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.807202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.814543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fd23-748e-482c-a60b-23375c66914a service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.944035 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.970371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.971542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.971834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.972115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.976447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.981513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.014745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d8a8-888e-4aa7-b8ee-0064ee48fe9a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.032755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1049/2064] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:26.033662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.037685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:26.064566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.065798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.065798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.066585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.068695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.071632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd68155-7f86-4263-b862-5d4323d4b68a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.072761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.087350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.088080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.088367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.088647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.089789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.095462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.127221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1016/2065] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:25 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:26.128816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.136565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1050/2066] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:26 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:26.137489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.150790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38f074-b47b-4c0a-a733-cf5fb526ece5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.153556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aaadf3-f370-47f2-ba67-20aecc5e20ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.158093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.159213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.159502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.159867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.160860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.163506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.165070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.165931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.166316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.166713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.168176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.179030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1051/2067] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.181116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.183754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.197593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe22d53-989a-46ec-8a73-4a43e2e43d27 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.221345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.222658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1017/2068] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.223179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.223468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.224148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.226378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.228487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.231871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.253328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1613c8b-64c8-4300-81a2-857077d254cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.266781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.267632 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.268120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.268492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.269345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.271070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.272988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1052/2069] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.275941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.300907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1018/2070] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.304512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.310539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9a83-1d40-4e02-9573-e3eda8faeb3f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.325260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.326012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.326396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.326684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.330123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85b2b0-bff0-4291-a793-6ede9940ba61 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.331702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.340348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.348623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.349951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.350456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.350941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] PATH_INFO: `/v3/projects/5cbd4516596f422badc5ddcb7445228b/users/b15d347b2290449e9767dee0eb509b42/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.377973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1053/2071] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.380634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.392072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbd4516596f422badc5ddcb7445228b, user_id=b15d347b2290449e9767dee0eb509b42, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.394647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.403848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460100-bc9e-4fe7-9583-6f195a53dd7a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.410581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.411861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.412454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.413311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] PATH_INFO: `/v3/projects/c3650a50b8da41208b6cb0d32d1e23e0/users/97fe7e1795454e7e828b360af277e388/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.429486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1019/2072] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:26 2022] PUT /identity/v3/projects/5cbd4516596f422badc5ddcb7445228b/users/b15d347b2290449e9767dee0eb509b42/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:26.430261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.437494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3650a50b8da41208b6cb0d32d1e23e0, user_id=97fe7e1795454e7e828b360af277e388, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.440907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.479381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1054/2073] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:26 2022] PUT /identity/v3/projects/c3650a50b8da41208b6cb0d32d1e23e0/users/97fe7e1795454e7e828b360af277e388/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:26.484233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.588630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30bd17-0b49-4c19-8953-7df248c792cc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.615935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b446-ba4a-463d-823a-034ebbe6d06e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.630303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.630870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.640061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.640650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.641997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.644281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ab0a7-2dd4-4d79-8d55-c0c949eaea2d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.669329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1055/2074] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:26.670797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.671188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.673429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.704798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] MFA Rules not processed for user `97fe7e1795454e7e828b360af277e388`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:26.706279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:26.709799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.710714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.711000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.711277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] PATH_INFO: `/v3/projects/27ad351e77294268843e41760f697341/users/658f3a35ad8046589d2d1f1d3e4b37c0/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.712850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:26.731132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ad351e77294268843e41760f697341, user_id=658f3a35ad8046589d2d1f1d3e4b37c0, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.733838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.757110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1020/2075] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:26 2022] PUT /identity/v3/projects/27ad351e77294268843e41760f697341/users/658f3a35ad8046589d2d1f1d3e4b37c0/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:26.758073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.760117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:26.796189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:26.798680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1056/2076] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:26.799786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:26.813878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17def14-87e2-445b-8689-2e81c775c8e7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.884311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3f9c3-5653-48f5-83c0-05b8125750a0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:26.891959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.894264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.894642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.895828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] PATH_INFO: `/v3/projects/84cfe2cd311c45678142b636545c831b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.897985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84cfe2cd311c45678142b636545c831b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.901484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.928789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:26.930005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.930296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.930914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] PATH_INFO: `/v3/projects/5cbd4516596f422badc5ddcb7445228b/users/b15d347b2290449e9767dee0eb509b42/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.933649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbd4516596f422badc5ddcb7445228b, user_id=b15d347b2290449e9767dee0eb509b42, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:26.936992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269520e9-77ff-44b8-aa06-df450bdedced admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:26.953366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1057/2077] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:26 2022] PUT /identity/v3/projects/5cbd4516596f422badc5ddcb7445228b/users/b15d347b2290449e9767dee0eb509b42/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:26.954988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e196e-67dc-4b5a-949c-96b31595e3c4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:26.955877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e196e-67dc-4b5a-949c-96b31595e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:26.956167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e196e-67dc-4b5a-949c-96b31595e3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:26.975904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84cfe2cd311c45678142b636545c831b', 'request_id': 'req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:26.988805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-639e196e-67dc-4b5a-949c-96b31595e3c4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:26.991314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-639e196e-67dc-4b5a-949c-96b31595e3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:26.993777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84cfe2cd311c45678142b636545c831b', 'request_id': 'req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.005832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84cfe2cd311c45678142b636545c831b', 'request_id': 'req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.040238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] The token cache is being invalidate because project 84cfe2cd311c45678142b636545c831b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:27.040667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.041460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.043228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1021/2078] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:26 2022] DELETE /identity/v3/projects/84cfe2cd311c45678142b636545c831b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:27.044503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.047446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.113021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1058/2079] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:27.113021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c513-666a-40d7-993d-a1d8baf1db5a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.113021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c513-666a-40d7-993d-a1d8baf1db5a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.113021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c513-666a-40d7-993d-a1d8baf1db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.161118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dc516-b03e-4ba2-8d93-71c528f0e0f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.173217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7551c513-666a-40d7-993d-a1d8baf1db5a None None] MFA Rules not processed for user `b15d347b2290449e9767dee0eb509b42`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:27.175107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7551c513-666a-40d7-993d-a1d8baf1db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.180014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.181981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.181981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.181981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.183989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.212285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:27.275243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1059/2080] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:27.276861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.277317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.277749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.285220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.286725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.300906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1022/2081] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:27.301687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.302159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:27.303134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.303436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.319888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1060/2082] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:27.319888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.320076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:27.321327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.339610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca945a-4ea0-49ef-b076-5b060ece211d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.346251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.347248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.347934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.348563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.361181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.362639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e47309-b504-4ced-bb6e-ae82322decbc service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.374514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1061/2083] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:27.375957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.376281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.376639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.395321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:27.396795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.407305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1062/2084] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:27.411166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.435790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa06bc-db7b-4261-9391-df1879126dba service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.439922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead9900-af4a-4325-8daa-266e4920e178 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.441851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.442865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.443081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.443377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.451945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.452745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.453058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.454208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.455032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:27.456319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.456640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.458540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.462496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.473853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1023/2085] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:27.476560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1063/2086] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:27.482626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.499362 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c31b-94a6-40a5-b0b4-55122aa7a93f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.506477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.507810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.507810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.508639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.509526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.512539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.561995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.562249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1024/2087] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:27.576930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b8ca-8afe-49db-b214-4e06670e3256 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.610309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.614266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.744405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.745017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.745017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.745211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.745781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.748067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.761644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4325-4bd9-461b-a2ae-e3bb7fd95a9b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.771183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.772433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.773102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.773385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] PATH_INFO: `/v3/users/4a55c4d6e33c4fa59f1107166e10b8c9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.777935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a55c4d6e33c4fa59f1107166e10b8c9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.781969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.801253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1064/2088] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:27.804362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.819624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dcf47-f1d1-4abd-aaa5-aa6a1ed05a3a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.835902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.837504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.837803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.838086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.838856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.841086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.859811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1065/2089] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:27.864328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.885604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a576e8-82d0-4942-9445-0b2f5b696a22 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.893973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a55c4d6e33c4fa59f1107166e10b8c9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.894990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a55c4d6e33c4fa59f1107166e10b8c9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.897398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.898828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.898828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.898828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.899423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.901107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.919225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1066/2090] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:27.920209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:27.922311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:27.935785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a55c4d6e33c4fa59f1107166e10b8c9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.937977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77bc0-cdf5-419f-b896-51e9499bc3c9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:27.945953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:27.946960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.947409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.947536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] PATH_INFO: `/v3/projects/780bee7d8b0b436eb3a84566022bb3f8/users/7eae149db5ba47d3b6f266a393f64bfe/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.947857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a55c4d6e33c4fa59f1107166e10b8c9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.949445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf298df7-e8fa-4a9e-ad60-5a7920748dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a55c4d6e33c4fa59f1107166e10b8c9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:27.977467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1025/2091] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:27 2022] DELETE /identity/v3/users/4a55c4d6e33c4fa59f1107166e10b8c9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:27.982689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:27.983197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:27.983539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:27.984047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780bee7d8b0b436eb3a84566022bb3f8, user_id=7eae149db5ba47d3b6f266a393f64bfe, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:27.992681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfbf79-dcbc-45aa-bd1f-2df649ec3489 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:27.998527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:28.029574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:28.033586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:28.035952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1067/2092] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:27 2022] PUT /identity/v3/projects/780bee7d8b0b436eb3a84566022bb3f8/users/7eae149db5ba47d3b6f266a393f64bfe/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:28.048175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.048750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.049102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.110412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] MFA Rules not processed for user `7eae149db5ba47d3b6f266a393f64bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:28.112836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:28.138638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:28.171800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1026/2093] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:28.196559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.199936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:28.223855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1068/2094] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:28.231443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.247860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e3c7-2471-48cc-932a-20a0b2b92454 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.260447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.263175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.263464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.263780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.267388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.273964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.345787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1069/2095] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:28 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:28.346884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.376192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbce2da-bbe2-463c-af68-1edbf1c92d37 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.467890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bca328-97d6-43c6-9614-7730ff6d67e6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.471115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.471939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.472469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.473066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.473582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.473989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.474401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.474714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.475569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.477676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.487844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.489460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.501313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1027/2096] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:28.508995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1070/2097] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:28 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:28.510776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.521215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978c7a-7e8e-4838-9f5b-7729a02be451 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.525676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.526327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.526635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.526934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.527650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.529271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.539645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1028/2098] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:28.543568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.554487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7afa6-c812-49e5-8b24-3b465c2f89ee admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.559573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.560602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.560888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.561159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] PATH_INFO: `/v3/projects/4123837592aa46fc924c323cc7adb4a9/users/0576562882a84e3f99eb124a3c522cb0/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.578916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4123837592aa46fc924c323cc7adb4a9, user_id=0576562882a84e3f99eb124a3c522cb0, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.581606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.595388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1071/2099] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:28 2022] PUT /identity/v3/projects/4123837592aa46fc924c323cc7adb4a9/users/0576562882a84e3f99eb124a3c522cb0/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:28.599328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.610370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844c73a-ebfc-452d-958c-06ae1417efcc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.664686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:28.672974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.673662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.673945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.674230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.674971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.676688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89034bb4-ed1d-4549-bed3-f7b788e24a56 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.687687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1029/2100] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:28.692070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.703469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5f2d-e292-410a-be79-b7b57112ec38 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:28.708958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:28.709953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.710235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.710507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] PATH_INFO: `/v3/projects/4123837592aa46fc924c323cc7adb4a9/users/0576562882a84e3f99eb124a3c522cb0/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.712493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4123837592aa46fc924c323cc7adb4a9, user_id=0576562882a84e3f99eb124a3c522cb0, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:28.715285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:28.725981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1072/2101] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:28 2022] PUT /identity/v3/projects/4123837592aa46fc924c323cc7adb4a9/users/0576562882a84e3f99eb124a3c522cb0/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:28.739787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287c4f5-1b50-4cd2-9a6a-d81f68c3082f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.740216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287c4f5-1b50-4cd2-9a6a-d81f68c3082f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.740566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287c4f5-1b50-4cd2-9a6a-d81f68c3082f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:28.770123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2287c4f5-1b50-4cd2-9a6a-d81f68c3082f None None] MFA Rules not processed for user `0576562882a84e3f99eb124a3c522cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:28.771509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2287c4f5-1b50-4cd2-9a6a-d81f68c3082f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:28.880339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1030/2102] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:28.892061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:28.895515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:28.980782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:28.981267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:28.981359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.026741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:29.032276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] MFA Rules not processed for user `658f3a35ad8046589d2d1f1d3e4b37c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:29.033512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.037404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94a3fd-cd9d-499f-8aa6-078bddfa6028 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.047742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.049267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.049267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.049267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.160839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1031/2103] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:29.162240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.164878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.184171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.186206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.202275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1073/2104] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:29.215007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.221940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.285724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a53fbb-3bb2-461e-9415-2144fb69efbf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.292031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.293432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.293707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.293976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] PATH_INFO: `/v3/projects/dff5e95f55664edfb76fdfe6e53eea40` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.295299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff5e95f55664edfb76fdfe6e53eea40)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.298068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.310360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:29.346752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d424c4-1757-44a6-9000-dd4df27dc8d6 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.355270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.356102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.356386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.357404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.358173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.360493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.392773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff5e95f55664edfb76fdfe6e53eea40', 'request_id': 'req-5744151f-eac6-4a57-8ea6-fad4edaef2b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.405096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1074/2105] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:29 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:29.406504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff5e95f55664edfb76fdfe6e53eea40', 'request_id': 'req-5744151f-eac6-4a57-8ea6-fad4edaef2b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.406697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.415916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff5e95f55664edfb76fdfe6e53eea40', 'request_id': 'req-5744151f-eac6-4a57-8ea6-fad4edaef2b6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.454281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] The token cache is being invalidate because project dff5e95f55664edfb76fdfe6e53eea40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:29.454281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.455405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.459981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1032/2106] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:29 2022] DELETE /identity/v3/projects/dff5e95f55664edfb76fdfe6e53eea40 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:29.460851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.462745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.529842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f7367-94c3-492b-a282-16490eae019a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.535800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.536860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.537338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.537824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.538730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.542679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.570264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1075/2107] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:29 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:29.574012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.582405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.596669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d228-5328-43c5-a3fd-068decb581e8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.607325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744151f-eac6-4a57-8ea6-fad4edaef2b6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.611308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.612035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.612469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.612754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.613915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.615431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.618850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.621392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.621942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.622496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] PATH_INFO: `/v3/users/b15d347b2290449e9767dee0eb509b42` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.624640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15d347b2290449e9767dee0eb509b42)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.627560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1076/2108] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:29.631580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.634140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.642324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476854a-6d19-40e0-bcf2-5591e9d7c707 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.647335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.649076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.649076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.649076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] PATH_INFO: `/v3/projects/0489aed38d164cac902639029accd34a/users/8d9e9669ed4e416c9b687c3320c2308a/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.652155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:29.667583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0489aed38d164cac902639029accd34a, user_id=8d9e9669ed4e416c9b687c3320c2308a, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.670341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.684937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1077/2109] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:29 2022] PUT /identity/v3/projects/0489aed38d164cac902639029accd34a/users/8d9e9669ed4e416c9b687c3320c2308a/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:29.688445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.705995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15d347b2290449e9767dee0eb509b42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.706694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15d347b2290449e9767dee0eb509b42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.723705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15d347b2290449e9767dee0eb509b42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.736883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15d347b2290449e9767dee0eb509b42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.738078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15d347b2290449e9767dee0eb509b42'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:29.757197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6099-25f6-4b89-bca8-b8525fb9e999 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.758689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1033/2110] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:29 2022] DELETE /identity/v3/users/b15d347b2290449e9767dee0eb509b42 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:29.833070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.833866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.834181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.834460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.835286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.837064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.849113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1078/2111] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:29.853188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:29.854644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:29.868017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b5fbb-4a4d-4638-8cce-d1fe1e1db075 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:29.873813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:29.874707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:29.874990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:29.875304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] PATH_INFO: `/v3/projects/0489aed38d164cac902639029accd34a/users/8d9e9669ed4e416c9b687c3320c2308a/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:29.877345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0489aed38d164cac902639029accd34a, user_id=8d9e9669ed4e416c9b687c3320c2308a, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:29.880014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402849c-87a8-49a8-b244-034c441527b3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:29.891386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1034/2112] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:29 2022] PUT /identity/v3/projects/0489aed38d164cac902639029accd34a/users/8d9e9669ed4e416c9b687c3320c2308a/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:30.206675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.209045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.221459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4086c-7f96-4ff0-ab86-3ae66a33f897 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.312837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.313944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.314107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.314413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] PATH_INFO: `/v3/projects/5cbd4516596f422badc5ddcb7445228b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.315617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbd4516596f422badc5ddcb7445228b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.317650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.359989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:30.383510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbd4516596f422badc5ddcb7445228b', 'request_id': 'req-b951b016-d2f6-41c4-9c68-bf9d2f838f20'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:30.389602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbd4516596f422badc5ddcb7445228b', 'request_id': 'req-b951b016-d2f6-41c4-9c68-bf9d2f838f20'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:30.398739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbd4516596f422badc5ddcb7445228b', 'request_id': 'req-b951b016-d2f6-41c4-9c68-bf9d2f838f20'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:30.412701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] The token cache is being invalidate because project 5cbd4516596f422badc5ddcb7445228b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:30.413688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:30.414446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b951b016-d2f6-41c4-9c68-bf9d2f838f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:30.420291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1079/2113] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:30 2022] DELETE /identity/v3/projects/5cbd4516596f422badc5ddcb7445228b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:30.423658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.424219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.424506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.432226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:30.451505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:30.453717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.525163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1035/2114] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:30.533094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.533417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.533683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.555073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:30.556320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.565544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1080/2115] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:30.569773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.579662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d5832-62c1-428c-af3f-041ef5fd4af0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.584823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.585555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.585905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.586249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.587048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:30.587432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.589253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.602150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1036/2116] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:30.607119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.616971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063519ec-62a2-42c9-bb66-8d912d40cd9c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.622902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.623569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.623893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.624173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.624864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.626836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b1ce0-3977-4147-acbb-c91e9b9f8759 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.655496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1081/2117] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:30.659883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.671728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ac61f-cec0-40b5-8a70-902826d5c92e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.704638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af63c10-0baf-4557-8b02-209489526411 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.704936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af63c10-0baf-4557-8b02-209489526411 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.705219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af63c10-0baf-4557-8b02-209489526411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.734101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4af63c10-0baf-4557-8b02-209489526411 None None] MFA Rules not processed for user `8d9e9669ed4e416c9b687c3320c2308a`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:30.735814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4af63c10-0baf-4557-8b02-209489526411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.746514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.748109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.748395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.748671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.749530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.751913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.777288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1037/2118] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:30.778957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.789629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66e1b1-32ff-4981-a491-1f41679486a0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.796548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.797214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.797494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.797796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.798528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.800154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.811221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1038/2119] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:30.814735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.820163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1082/2120] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:30.824954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1f4e-2262-48ed-987c-4c0bccffac97 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.829648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.830366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.830582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.830858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.831593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.833295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.834144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af63c10-0baf-4557-8b02-209489526411 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.836724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4af63c10-0baf-4557-8b02-209489526411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.845257 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1039/2121] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:37:30.851301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:30.862752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0fce3-b0f0-4bee-8563-25a87840613f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.868479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.870826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.871387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.871704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] PATH_INFO: `/v3/projects/a7e95e06a32345acbc83c1de50c96cb4/users/24d5d1a5f8464d0db0d280c297335459/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.891687 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e95e06a32345acbc83c1de50c96cb4, user_id=24d5d1a5f8464d0db0d280c297335459, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:30.894410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0180-4391-4803-8011-6c4e9dd29164 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:30.907706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af63c10-0baf-4557-8b02-209489526411 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:30.908528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1040/2122] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:30 2022] PUT /identity/v3/projects/a7e95e06a32345acbc83c1de50c96cb4/users/24d5d1a5f8464d0db0d280c297335459/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:30.919329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.919627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.919928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.938769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:30.947139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] MFA Rules not processed for user `24d5d1a5f8464d0db0d280c297335459`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:30.948435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.963214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:30.964428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:30.964717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:30.964991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:30.966788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:30.970368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:31.017282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1041/2123] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:31.021712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.028489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.029940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.039886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1083/2124] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:31.090293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae0ef3-6529-436a-9e92-a4a536251f7c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.095311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.096094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.096379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.096655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.097352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.099256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.130255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1042/2125] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:31 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:31.131492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.143496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdac6b6-5bd7-4842-b84d-83ac69b51bd0 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.213139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.213876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.214593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.214885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.215569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.217572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.238650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1084/2126] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:31 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:31.242425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.254136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b04cc-5ba0-4bf9-8a1b-44bb26975696 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.259446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.260245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.260602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.262077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.262898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.265194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.278099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1043/2127] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:31.282271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.293484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715b285-92b1-4db5-ab2a-0af1256ea03a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.299514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.300430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.300714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.300992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] PATH_INFO: `/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65/users/b9403a0062ec48428f38b6ae03a599eb/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.318362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb5e4cb38ac4657a25cd27a2566ed65, user_id=b9403a0062ec48428f38b6ae03a599eb, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.321171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.336680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1085/2128] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:31 2022] PUT /identity/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65/users/b9403a0062ec48428f38b6ae03a599eb/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:31.341320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.351995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754747fc-8fda-4fe4-a57f-dde8248624f1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.432803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.435310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.435669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.436029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.436851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.438498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8196d2c-c535-40a2-a50a-3f0977ba20f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.454592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1044/2129] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:31.459176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.470680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b22e-4885-4ab6-ad74-805e14b1933a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.475420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.476388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.476674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.476952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] PATH_INFO: `/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65/users/b9403a0062ec48428f38b6ae03a599eb/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.478992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb5e4cb38ac4657a25cd27a2566ed65, user_id=b9403a0062ec48428f38b6ae03a599eb, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.481731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.494218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1086/2130] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:31 2022] PUT /identity/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65/users/b9403a0062ec48428f38b6ae03a599eb/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:31.505259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.505552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.505826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.536327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] MFA Rules not processed for user `b9403a0062ec48428f38b6ae03a599eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:31.537588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:31.559266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:31.610408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1045/2131] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:31.622038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.623836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:31.664034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:31.682076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51d138-e214-455c-9a15-161ec9cca44d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.686235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.686878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.687162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.687441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.696883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.698392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.708805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1087/2132] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:31.785933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:31.795375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5efcd-648d-406b-a9a4-251f61938bb6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:31.798953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:31.799590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:31.799906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:31.800197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:31.856551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:31.857916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:31.866981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1046/2133] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:35.583657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:35.592436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb7a9d-1f47-419a-90d5-b4da29f3467b service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:35.596038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:35.596698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:35.596982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:35.597258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:35.657678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:35.659098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120929-6e96-4629-8167-254da6e52a75 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:35.669321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1088/2134] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:36.806568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:36.808039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:36.873916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63266d6f-d909-460b-b24f-e4545103095c service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:36.878765 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:36.879806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:36.880088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:36.880359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] PATH_INFO: `/v3/users/97fe7e1795454e7e828b360af277e388` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:36.881537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fe7e1795454e7e828b360af277e388)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:36.883807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:36.931095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fe7e1795454e7e828b360af277e388'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:36.931981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fe7e1795454e7e828b360af277e388'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:36.941582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:36.948114 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fe7e1795454e7e828b360af277e388'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:36.955870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fe7e1795454e7e828b360af277e388'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:36.956580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fe7e1795454e7e828b360af277e388'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:36.971611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1047/2135] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:36 2022] DELETE /identity/v3/users/97fe7e1795454e7e828b360af277e388 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:36.976115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:36.976959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:36.977378 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.003542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:37.004777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:37.075607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1089/2136] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:37.086442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.087989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:37.149553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb6910-e72c-44cd-bb51-e9966e1de920 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.153838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.154489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.154801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.155076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.164627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.166044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e46ad-bdc1-4689-b3de-d818d3cea6d0 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.175544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1048/2137] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:37.452475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.453911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:37.466595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed321c4d-1354-4a1a-b43b-c585f28bb7fc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.471424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.472404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.472757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.473129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] PATH_INFO: `/v3/projects/c3650a50b8da41208b6cb0d32d1e23e0` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.474352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3650a50b8da41208b6cb0d32d1e23e0)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.476962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.524359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3650a50b8da41208b6cb0d32d1e23e0', 'request_id': 'req-327a2ad6-e1ec-4edf-90bd-c4fad829820b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:37.529722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3650a50b8da41208b6cb0d32d1e23e0', 'request_id': 'req-327a2ad6-e1ec-4edf-90bd-c4fad829820b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:37.536624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3650a50b8da41208b6cb0d32d1e23e0', 'request_id': 'req-327a2ad6-e1ec-4edf-90bd-c4fad829820b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:37.538594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:37.549286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] The token cache is being invalidate because project c3650a50b8da41208b6cb0d32d1e23e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:37.549578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:37.550228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-327a2ad6-e1ec-4edf-90bd-c4fad829820b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:37.551837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1090/2138] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:37 2022] DELETE /identity/v3/projects/c3650a50b8da41208b6cb0d32d1e23e0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:37:37.556942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.557199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.558010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.575805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:37.576996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:37.644102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1049/2139] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:37.652913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.653205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.653487 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.672427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:37.673888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:37.681980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1091/2140] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:37.686619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.696284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43938af4-d6d7-49c8-b4f6-92b3d93aea9f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.701471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.702129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.702412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.702688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.703411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:37.708577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.708577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.718502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1050/2141] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:37:37.722695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.732316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4313e-8a47-4a65-8d76-6e5a511897fb None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.737047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.737701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.737984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.738260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.738962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.741112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.771498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1092/2142] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:37 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:37.772880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.783623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91d9f-182d-4b70-bdb7-9ed03805afdd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.849391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.850088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.850366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.850642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.851339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.853424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.864708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:37.876677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1051/2143] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:37 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:37.880959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.891367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dedfb4-a436-43b3-b55a-01163d060c20 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.896175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.896835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.897117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.897394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.898121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.899706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.911796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1093/2144] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:37.915153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.925738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee57fd8-1497-4ab9-904c-cdbc9734a947 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.930577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.931420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.931707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.932020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.933313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.934899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f5e4e-276a-4a72-be1b-82b14e2dacb5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.945449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1052/2145] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:37.949618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:37.959969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a67014-26d1-492e-8850-a3edfe257d26 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:37.965223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:37.966106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:37.966387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:37.966659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] PATH_INFO: `/v3/projects/c7a047de6ac04969b66d01f16f257458/users/92597e9431bb4004b8a75220f4a299bf/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:37.982340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a047de6ac04969b66d01f16f257458, user_id=92597e9431bb4004b8a75220f4a299bf, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:37.985030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:37.999970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1094/2146] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:37 2022] PUT /identity/v3/projects/c7a047de6ac04969b66d01f16f257458/users/92597e9431bb4004b8a75220f4a299bf/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:38.010078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:38.010381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:38.011321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:38.038668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] MFA Rules not processed for user `92597e9431bb4004b8a75220f4a299bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:38.039928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:38.121689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1053/2147] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:38.132279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:38.134052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:38.201734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4d2bf-0891-46e0-8142-12866a5dc8f0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:38.206386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:38.207091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:38.207373 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:38.207648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:38.217818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:38.219252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:38.228248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1095/2148] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:41.644459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:41.654033 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14712a-0b2b-49d7-b5cc-83507ccb287f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:41.658533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:41.659180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:41.659472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:41.659740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:41.661688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:41.732131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:41.733559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195d99-fbbc-4e82-bf0b-e8f290d000d3 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:41.743485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1054/2149] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:42.875184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:42.876691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:42.959518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a5d91-18b5-4ebe-92c4-4d05bd7224a5 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:42.964142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:42.965012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:42.965390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:42.965558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] PATH_INFO: `/v3/users/3c0c279fe9ef4148be77d332aae6bafb` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:42.966752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0c279fe9ef4148be77d332aae6bafb)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:42.968699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.008391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:43.020706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0c279fe9ef4148be77d332aae6bafb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.033497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0c279fe9ef4148be77d332aae6bafb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.034183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0c279fe9ef4148be77d332aae6bafb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.034797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0c279fe9ef4148be77d332aae6bafb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.041719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0c279fe9ef4148be77d332aae6bafb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.055285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1096/2150] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:42 2022] DELETE /identity/v3/users/3c0c279fe9ef4148be77d332aae6bafb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:43.061528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.062009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.062149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.080315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:43.081514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:43.155952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1055/2151] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:43.158784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.160256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:43.232858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ff9fa-e3d3-48bd-84c4-c4ab444ec19e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.237421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.241640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.241640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.241640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.249828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.250878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e1b2f-738f-4487-9c11-2eadd333e1b3 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.261748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1097/2152] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:43.542423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.543868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:43.555678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803747a6-caaa-4065-a347-1f8d849894f1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.560871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.561737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.562021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.562292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] PATH_INFO: `/v3/projects/68125869b066435cb99a781bc3eba8f0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.563479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68125869b066435cb99a781bc3eba8f0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.565442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.568027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:43.612536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68125869b066435cb99a781bc3eba8f0', 'request_id': 'req-fab013f1-a61f-446e-bf35-e26a2ee0c991'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.617953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68125869b066435cb99a781bc3eba8f0', 'request_id': 'req-fab013f1-a61f-446e-bf35-e26a2ee0c991'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.624769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68125869b066435cb99a781bc3eba8f0', 'request_id': 'req-fab013f1-a61f-446e-bf35-e26a2ee0c991'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.637715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] The token cache is being invalidate because project 68125869b066435cb99a781bc3eba8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:43.637997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.638629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fab013f1-a61f-446e-bf35-e26a2ee0c991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:43.640192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1056/2153] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:43 2022] DELETE /identity/v3/projects/68125869b066435cb99a781bc3eba8f0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:43.645058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.645347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.645620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.663448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:43.664713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:43.723840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1098/2154] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:43.732643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.732930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.733206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.752068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:43.753267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:43.761716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1057/2155] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:43.765783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.775371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9f4ce-a3d6-462e-b740-3892f6b75430 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.779793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.780457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.780745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.781023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.781748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:43.782057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.783641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.795096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1099/2156] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:43.799129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.808763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a91ccd-ab94-414e-93d0-c3f01c5f80af None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.813375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.814024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.814306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.814580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.815266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.817184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.842449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1058/2157] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:43 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:43.846504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.857016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0b71-e950-4b2d-8525-c6c9b961059b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.898887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:43.914890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.915555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.915872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.916151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.917368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.919387 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.940819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1100/2158] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:43 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:43.945071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:43.966095 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c721c-f7e7-410c-91d7-1c8d43babeb2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:43.970845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:43.971606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:43.971986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:43.972328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:43.973124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:43.974776 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:43.986453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1059/2159] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:43.990554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:44.002025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a300c6-bfb5-41d0-aca0-8043aa96ad78 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:44.007090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:44.007781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:44.008068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:44.008345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:44.009080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:44.010729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f039d54-1c05-409c-b14d-cb8d48e28740 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:44.030794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1101/2160] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:44.034777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:44.045596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac96d3-a5f4-4c00-8993-1541894d5cba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:44.050271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:44.051148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:44.051430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:44.051705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] PATH_INFO: `/v3/projects/f9124a7a7d7e498b9f7c28671200e203/users/0f6f9343977343d78848c52d103255f5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:44.066707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9124a7a7d7e498b9f7c28671200e203, user_id=0f6f9343977343d78848c52d103255f5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:44.069416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:44.084225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1060/2161] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:44 2022] PUT /identity/v3/projects/f9124a7a7d7e498b9f7c28671200e203/users/0f6f9343977343d78848c52d103255f5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:44.089470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe14-bbd0-4abe-b02c-790793e48728 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:44.089816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe14-bbd0-4abe-b02c-790793e48728 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:44.090098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989fe14-bbd0-4abe-b02c-790793e48728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:44.108101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8989fe14-bbd0-4abe-b02c-790793e48728 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:44.109288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8989fe14-bbd0-4abe-b02c-790793e48728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:44.169291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1102/2162] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:44.178997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:44.180703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:44.241585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd547c3d-3448-41ba-8aab-aeffc9d02799 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:44.248826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:44.249492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:44.249775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:44.250053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:44.260098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:44.261616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:44.275105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1061/2163] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:44.892652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:44.892941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:44.893220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:44.920034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] MFA Rules not processed for user `0f6f9343977343d78848c52d103255f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:44.921569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:44.988319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1103/2164] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:44.999877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:45.001402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:45.037248 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:45.062008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97727662-cb21-4b08-a417-c8ad92dfce98 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:45.065586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:45.066235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:45.066518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:45.066799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:45.075956 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:45.077323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e19af-702a-44fd-a295-78c2402c6317 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:45.086102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1062/2165] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:47.660122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:47.661642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:47.675848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94363da9-b42d-4c95-afbe-3784fe7c9517 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:47.680405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:47.681298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:47.681583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:47.681864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] PATH_INFO: `/v3/users/24d5d1a5f8464d0db0d280c297335459` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:47.683633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d5d1a5f8464d0db0d280c297335459)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:47.685608 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:47.733488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d5d1a5f8464d0db0d280c297335459'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:47.734589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:47.749032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d5d1a5f8464d0db0d280c297335459'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:47.749736 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d5d1a5f8464d0db0d280c297335459'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:47.750349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d5d1a5f8464d0db0d280c297335459'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:47.757177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d5d1a5f8464d0db0d280c297335459'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:47.769321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1104/2166] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:47 2022] DELETE /identity/v3/users/24d5d1a5f8464d0db0d280c297335459 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:47.773742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:47.774193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:47.774335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:47.790958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:47.792713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:47.849917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1063/2167] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:47.860245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:47.861628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:47.919134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bba2eb-f5a3-444b-95f9-f114db34290a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:47.922743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:47.923391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:47.923678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:47.923986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:47.933558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:47.935053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:47.945082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1105/2168] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:48.209109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:48.210596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:48.223950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224d7f8-550e-4437-b8a7-46d2e42efc24 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:48.228655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:48.230196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:48.230548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:48.230898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] PATH_INFO: `/v3/projects/a7e95e06a32345acbc83c1de50c96cb4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:48.232189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e95e06a32345acbc83c1de50c96cb4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:48.234168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:48.280814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e95e06a32345acbc83c1de50c96cb4', 'request_id': 'req-27d3909b-53d4-4a82-a9e4-808c4a86f99c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.288377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e95e06a32345acbc83c1de50c96cb4', 'request_id': 'req-27d3909b-53d4-4a82-a9e4-808c4a86f99c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.296212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e95e06a32345acbc83c1de50c96cb4', 'request_id': 'req-27d3909b-53d4-4a82-a9e4-808c4a86f99c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.311050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] The token cache is being invalidate because project a7e95e06a32345acbc83c1de50c96cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:48.311336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.311987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.313554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1064/2169] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:48 2022] DELETE /identity/v3/projects/a7e95e06a32345acbc83c1de50c96cb4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:48.318788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:48.329023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cee8b-cb0b-46f4-806b-83ae166631dc service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:48.394282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:48.395157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:48.395444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:48.395726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] PATH_INFO: `/v3/users/b9403a0062ec48428f38b6ae03a599eb` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:48.397088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9403a0062ec48428f38b6ae03a599eb)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:48.398964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:48.447564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9403a0062ec48428f38b6ae03a599eb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.459540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9403a0062ec48428f38b6ae03a599eb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.460150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9403a0062ec48428f38b6ae03a599eb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.460757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9403a0062ec48428f38b6ae03a599eb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.465628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:48.469297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-32f2e022-932a-41c2-896f-59bc0cba4dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9403a0062ec48428f38b6ae03a599eb'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:48.484826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1106/2170] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:48 2022] DELETE /identity/v3/users/b9403a0062ec48428f38b6ae03a599eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:48.892412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:48.894451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:48.909089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3909b-53d4-4a82-a9e4-808c4a86f99c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:48.966503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:48.973326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:48.974278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:48.974736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:48.975171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] PATH_INFO: `/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:48.976562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb5e4cb38ac4657a25cd27a2566ed65)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:48.978479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.024197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb5e4cb38ac4657a25cd27a2566ed65', 'request_id': 'req-d1b3c306-3094-4af6-83f3-45385ab3b855'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:49.029699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb5e4cb38ac4657a25cd27a2566ed65', 'request_id': 'req-d1b3c306-3094-4af6-83f3-45385ab3b855'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:49.035817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb5e4cb38ac4657a25cd27a2566ed65', 'request_id': 'req-d1b3c306-3094-4af6-83f3-45385ab3b855'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:49.047879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] The token cache is being invalidate because project 4fb5e4cb38ac4657a25cd27a2566ed65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:49.048167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:49.048790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d1b3c306-3094-4af6-83f3-45385ab3b855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:49.050298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1065/2171] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:48 2022] DELETE /identity/v3/projects/4fb5e4cb38ac4657a25cd27a2566ed65 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:49.055155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.055634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.055720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.074472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:49.075665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:49.133926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1107/2172] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:49.141658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.141951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.142252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.161271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:49.162459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:49.170788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1066/2173] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:49.174859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.185187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f3d2c-e007-423e-8dac-6af462dc1a54 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.189685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.190338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.190622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.190901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.191632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:49.191978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.193568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.206235 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1108/2174] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:49.210453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.219794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f04c-5224-43de-af72-44fe79baf72d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.224339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.224985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.225266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.225542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.226240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.228153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.254540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1067/2175] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:49 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:49.258527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.268446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21b8c-3636-4362-b575-80fdee7334fc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.324344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.325008 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.325288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.325562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.326245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.328210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.350587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1109/2176] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:49 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:49.351841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.362674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca19c4-b75d-4328-be04-9d8ccd459f2c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.367347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.368039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.368325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.368602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.369324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.371730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.381993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1068/2177] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:49.385906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.395564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170664cb-1a29-4bc2-bcfc-05deab345c25 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.400154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.400800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.401111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.401390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.402350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.404042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804892b5-413b-48a7-b5b2-92063fa35c39 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.413826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1110/2178] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:49.418293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.432945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f16c3c-b1b4-42e7-b92f-a0945c9cfe81 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.437843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.438763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.439049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.439324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] PATH_INFO: `/v3/projects/9e6cd40fc3354dfcb3821979cd6c9889/users/8fd0d94821f640a1949424c64dd71b65/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.454391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6cd40fc3354dfcb3821979cd6c9889, user_id=8fd0d94821f640a1949424c64dd71b65, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.457076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.470449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1069/2179] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:49 2022] PUT /identity/v3/projects/9e6cd40fc3354dfcb3821979cd6c9889/users/8fd0d94821f640a1949424c64dd71b65/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:49.481543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f56df-5897-403c-a242-d71b4cae5d2d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.481837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f56df-5897-403c-a242-d71b4cae5d2d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.482120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f56df-5897-403c-a242-d71b4cae5d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.508871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-459f56df-5897-403c-a242-d71b4cae5d2d None None] MFA Rules not processed for user `8fd0d94821f640a1949424c64dd71b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:49.510069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-459f56df-5897-403c-a242-d71b4cae5d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:49.552641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:49.578211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1111/2180] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:49.588927 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.590660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:49.647315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa56b-29bc-4e90-b2bc-360e33e33a57 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.651037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.651679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.652021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.652297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.660627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.662052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7b5ef-65ef-4865-a26f-ae1dba66a3d0 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.670802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1070/2181] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:56.022548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.023064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.023320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.043620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] MFA Rules not processed for user `0f6f9343977343d78848c52d103255f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:56.045013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:56.104686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1112/2182] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:59.667163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:03.628253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9e49-e4a4-4559-9132-ece36fb400ef None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:03.628883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9e49-e4a4-4559-9132-ece36fb400ef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:03.628883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9e49-e4a4-4559-9132-ece36fb400ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:03.646873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea6d9e49-e4a4-4559-9132-ece36fb400ef None None] MFA Rules not processed for user `8fd0d94821f640a1949424c64dd71b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:03.649116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea6d9e49-e4a4-4559-9132-ece36fb400ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:03.656184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1071/2183] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:06.097258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=33 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:13.641109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:33.576583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:33.578783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:33.684714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1b39a-5715-4166-a798-4af6c67111a3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:33.690390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:33.691484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:33.691992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:33.692775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] PATH_INFO: `/v3/users/92597e9431bb4004b8a75220f4a299bf` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:33.694402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=92597e9431bb4004b8a75220f4a299bf)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:33.696654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:33.764362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92597e9431bb4004b8a75220f4a299bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:33.779544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92597e9431bb4004b8a75220f4a299bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:33.780076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92597e9431bb4004b8a75220f4a299bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:33.780813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92597e9431bb4004b8a75220f4a299bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:33.789044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92597e9431bb4004b8a75220f4a299bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:33.812073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1113/2184] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:33 2022] DELETE /identity/v3/users/92597e9431bb4004b8a75220f4a299bf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:33.816471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:33.816944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:33.817051 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:33.836660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:33.839314 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:33.922800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1072/2185] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:33.933961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:33.936802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:34.037567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9aaf-ffc3-45f6-802c-beceee1a83fc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.046814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.047554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.047870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.048150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.060974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.062422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011e66-e75c-4749-ad3e-ec9bf2685e21 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.073776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1114/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:34.380146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:34.381780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:34.399300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00779d-53c9-4a3b-9a29-b35b63de7e6c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.405741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.406803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.407089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.407363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] PATH_INFO: `/v3/projects/c7a047de6ac04969b66d01f16f257458` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.409858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a047de6ac04969b66d01f16f257458)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.412104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.476146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a047de6ac04969b66d01f16f257458', 'request_id': 'req-b1e08202-4804-41e8-a656-e5b8b11ed787'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:34.483240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a047de6ac04969b66d01f16f257458', 'request_id': 'req-b1e08202-4804-41e8-a656-e5b8b11ed787'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:34.489713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a047de6ac04969b66d01f16f257458', 'request_id': 'req-b1e08202-4804-41e8-a656-e5b8b11ed787'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:34.505451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] The token cache is being invalidate because project c7a047de6ac04969b66d01f16f257458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:34.505901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:34.506763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1e08202-4804-41e8-a656-e5b8b11ed787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:34.508771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1073/2187] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:34 2022] DELETE /identity/v3/projects/c7a047de6ac04969b66d01f16f257458 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:34.513722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.514015 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.514322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.537383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:34.539241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:34.565351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:34.637010 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1115/2188] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:34.646196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.646488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.647073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.691508 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:34.692978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:34.706673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1074/2189] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:34.707980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:34.720216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2ad6-b6d0-4434-8c08-60bfd317a521 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.726669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.727282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.727568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.727920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.728708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:34.729020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.730629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.750688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1116/2190] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:38:34.752224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:34.768951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8624df6-d6d2-4b8e-9574-0caed42e38e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.776920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.777700 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.778283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.778568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.779315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.783011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.815897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1075/2191] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:34 2022] POST /identity/v3/projects => generated 397 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:34.817169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:34.830553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c4adb-f457-4aa2-b52e-7f787303852d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.914122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.915048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.915429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.915829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.917368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.922755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.956575 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1117/2192] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:34 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:34.958140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:34.970603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2a10-571a-4373-8f62-f33fd5c09aab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:34.976891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:34.978094 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:34.978587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:34.979549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:34.980488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:34.983025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:34.996609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1076/2193] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:35.000396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:35.011794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41635f9-6469-4e22-90fd-79dd89569b59 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:35.017084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:35.017787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:35.018574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:35.018574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:35.019153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:35.021127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad938595-b414-45f1-98a0-fe5dbe997175 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:35.037831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1118/2194] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:35.039057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:35.064623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054af82-58f1-4efa-9ab2-fea41739b485 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:35.081779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:35.082953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:35.083534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:35.084302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] PATH_INFO: `/v3/projects/68560c74ae4b428f90be95b64ca55c3f/users/84159744669a4f328f860fbc45822969/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:35.104626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68560c74ae4b428f90be95b64ca55c3f, user_id=84159744669a4f328f860fbc45822969, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:35.110088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:35.131307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1077/2195] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:35 2022] PUT /identity/v3/projects/68560c74ae4b428f90be95b64ca55c3f/users/84159744669a4f328f860fbc45822969/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:35.142588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:35.143093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:35.143443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:35.174132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] MFA Rules not processed for user `84159744669a4f328f860fbc45822969`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:35.175645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:35.254128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1119/2196] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:35.260876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:35.262630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:35.285402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:35.326554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da41ee-4731-4112-a3f0-51223c1a684c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:35.330427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:35.330810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:35.331089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:35.331365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:35.340550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:35.341979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f48af-fc48-40e2-b55f-e5415573eab7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:35.351540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1078/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:36.212969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:36.214928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:36.240129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:36.292910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f3b45-c1b6-4848-a9e1-b7b08e706a96 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:36.299693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:36.300778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:36.301158 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:36.301536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] PATH_INFO: `/v3/users/7eae149db5ba47d3b6f266a393f64bfe` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:36.303134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eae149db5ba47d3b6f266a393f64bfe)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:36.305391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:36.369134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eae149db5ba47d3b6f266a393f64bfe'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:36.399794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eae149db5ba47d3b6f266a393f64bfe'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:36.402558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eae149db5ba47d3b6f266a393f64bfe'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:36.403321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eae149db5ba47d3b6f266a393f64bfe'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:36.424503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eae149db5ba47d3b6f266a393f64bfe'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:36.443912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1120/2198] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:36 2022] DELETE /identity/v3/users/7eae149db5ba47d3b6f266a393f64bfe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:36.446117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:36.447175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:36.447557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:36.466152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:36.467372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:36.541990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1079/2199] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:36.551576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:36.553090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:36.650345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8e0fb-ce45-4f20-ab5f-4c9c393d6e37 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:36.658130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:36.658927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:36.659215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:36.659492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:36.670817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:36.672240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:36.690535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1121/2200] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:37.218833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:37.220305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:37.235819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7090a32-1b03-450b-b55d-8558bf314f92 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:37.240947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:37.241853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:37.242139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:37.242418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] PATH_INFO: `/v3/projects/780bee7d8b0b436eb3a84566022bb3f8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:37.243720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780bee7d8b0b436eb3a84566022bb3f8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:37.245743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:37.316930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780bee7d8b0b436eb3a84566022bb3f8', 'request_id': 'req-8564cc2e-efc9-42b6-8728-33f0e4c98a38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.324150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780bee7d8b0b436eb3a84566022bb3f8', 'request_id': 'req-8564cc2e-efc9-42b6-8728-33f0e4c98a38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.330721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780bee7d8b0b436eb3a84566022bb3f8', 'request_id': 'req-8564cc2e-efc9-42b6-8728-33f0e4c98a38'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.346854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] The token cache is being invalidate because project 780bee7d8b0b436eb3a84566022bb3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:37.347102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.347878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.349511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1080/2201] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:37 2022] DELETE /identity/v3/projects/780bee7d8b0b436eb3a84566022bb3f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:37.353999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:37.369815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfb5a-f8b1-4bdc-8c26-502a22caff5b service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:37.432847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:37.458729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:37.459619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:37.459949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:37.460226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] PATH_INFO: `/v3/users/0576562882a84e3f99eb124a3c522cb0` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:37.461475 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0576562882a84e3f99eb124a3c522cb0)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:37.464188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:37.560162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0576562882a84e3f99eb124a3c522cb0'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.585385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0576562882a84e3f99eb124a3c522cb0'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.586259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0576562882a84e3f99eb124a3c522cb0'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.587630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0576562882a84e3f99eb124a3c522cb0'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.596653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f6464694-79e9-4f2c-a888-a0661d217dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0576562882a84e3f99eb124a3c522cb0'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:37.611666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1122/2202] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:37 2022] DELETE /identity/v3/users/0576562882a84e3f99eb124a3c522cb0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:38.033784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.036116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.060747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564cc2e-efc9-42b6-8728-33f0e4c98a38 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.117340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:38.137126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.138000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.138285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.138561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] PATH_INFO: `/v3/projects/4123837592aa46fc924c323cc7adb4a9` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.140043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4123837592aa46fc924c323cc7adb4a9)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.142242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.214957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4123837592aa46fc924c323cc7adb4a9', 'request_id': 'req-134418ea-c7cf-4909-bfa1-5166a3d2deb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.221857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4123837592aa46fc924c323cc7adb4a9', 'request_id': 'req-134418ea-c7cf-4909-bfa1-5166a3d2deb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.229891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4123837592aa46fc924c323cc7adb4a9', 'request_id': 'req-134418ea-c7cf-4909-bfa1-5166a3d2deb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.242568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] The token cache is being invalidate because project 4123837592aa46fc924c323cc7adb4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:38.242841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.243474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-134418ea-c7cf-4909-bfa1-5166a3d2deb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.245067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1081/2203] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:38 2022] DELETE /identity/v3/projects/4123837592aa46fc924c323cc7adb4a9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:38.251829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.251829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.251829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.270449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:38.271636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.348476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1123/2204] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:38.357153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd91b2f-2535-447e-8785-ed7173e53055 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.357460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd91b2f-2535-447e-8785-ed7173e53055 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.357758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd91b2f-2535-447e-8785-ed7173e53055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.383647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1bd91b2f-2535-447e-8785-ed7173e53055 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:38.384950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1bd91b2f-2535-447e-8785-ed7173e53055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.395860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1082/2205] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:38.398307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.410786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3dc2-462f-4c7e-97b3-a21c724f8920 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.427604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.428431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.429078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.429685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.430383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:38.430747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.432714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.453826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1124/2206] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:38:38.455529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4ba4-cbc4-4c30-873c-d127955e9713 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.455848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4ba4-cbc4-4c30-873c-d127955e9713 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.456016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.456157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4ba4-cbc4-4c30-873c-d127955e9713 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.457893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1083/2207] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:38:38.468071 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a33800-a47a-474d-a869-cd61e5dfe016 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.474523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.475188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.475661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.475961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.476653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.479856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.487307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.488226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.488740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.517703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:38.519313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.526366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1125/2208] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:38 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:38.531638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.637977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:38.659617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1084/2209] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:38.663554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.672995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.676774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33f3a0-437d-42b5-8e2a-cb2f3e790f56 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.682287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.683052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.683401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.683748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.686047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.688088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.702083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66b7d3-6472-4cfa-ac9f-d726d176b2c6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.707132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.708195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.708566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.708900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] PATH_INFO: `/v3/users/e4fc15fb3fc74eb597cdcf513cf22b2d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.710219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fc15fb3fc74eb597cdcf513cf22b2d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.713962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.724118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1126/2210] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:38 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:38.724118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.732839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.806552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fc15fb3fc74eb597cdcf513cf22b2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.807850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fc15fb3fc74eb597cdcf513cf22b2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.823970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:38.828392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fc15fb3fc74eb597cdcf513cf22b2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.842584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fc15fb3fc74eb597cdcf513cf22b2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.843243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fc15fb3fc74eb597cdcf513cf22b2d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:38.847652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4456b-b433-460b-a72c-eaa336361e7e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.859105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:38.867376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1085/2211] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:38 2022] DELETE /identity/v3/users/e4fc15fb3fc74eb597cdcf513cf22b2d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:38.867376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:38.871175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:38.872243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:38.872355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:38.877483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.878542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:38.897038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:38.898945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:38.910804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1127/2212] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:38.993023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb7e7-2278-49e7-aacd-4c61ce9f68b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:38.999348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.000082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.000366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.000641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.001381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.002991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.019106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1086/2213] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:39.020933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.034273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd5f57-3705-4b7f-ab1e-8e430906c680 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.041447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.042258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.042654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.043030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.043919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.045622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.059208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1128/2214] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:39.063467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.075545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcabfc-b7c7-4d9d-8376-68a7874a82ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.081123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.082146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.082533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.082928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] PATH_INFO: `/v3/projects/9a471a0dcde9474cb908566dc35969ca/users/4a1b9c8c4b95435793d501eea0c8580c/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.100941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a471a0dcde9474cb908566dc35969ca, user_id=4a1b9c8c4b95435793d501eea0c8580c, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.104189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228da46a-b863-4b40-a29a-a9f785459d34 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.118853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1087/2215] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:39 2022] PUT /identity/v3/projects/9a471a0dcde9474cb908566dc35969ca/users/4a1b9c8c4b95435793d501eea0c8580c/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:39.130246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.132672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.132963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.133243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.133508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.167535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7a729-c10f-4550-99a4-7305cfda3b78 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.182235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] MFA Rules not processed for user `4a1b9c8c4b95435793d501eea0c8580c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:39.183478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.271844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.273195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.273633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.274058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] PATH_INFO: `/v3/projects/5f7933bab51b4f9d9ed727c54f34e736` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.275544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7933bab51b4f9d9ed727c54f34e736)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.277942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.281926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1088/2216] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:39.295696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.297552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.323317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:39.340544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7933bab51b4f9d9ed727c54f34e736', 'request_id': 'req-762cdf7d-a58e-4168-9ef3-558520f146c6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:39.347437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:39.347695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7933bab51b4f9d9ed727c54f34e736', 'request_id': 'req-762cdf7d-a58e-4168-9ef3-558520f146c6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:39.356841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7933bab51b4f9d9ed727c54f34e736', 'request_id': 'req-762cdf7d-a58e-4168-9ef3-558520f146c6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:39.373133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] The token cache is being invalidate because project 5f7933bab51b4f9d9ed727c54f34e736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:39.373551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:39.374381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-762cdf7d-a58e-4168-9ef3-558520f146c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:39.376454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1129/2217] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:39 2022] DELETE /identity/v3/projects/5f7933bab51b4f9d9ed727c54f34e736 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:39.381398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd137e-1533-4129-9537-ebc4c058c2f9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.381704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd137e-1533-4129-9537-ebc4c058c2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.381988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd137e-1533-4129-9537-ebc4c058c2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.391368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede195b1-f74f-46d6-afba-3da35c2ae6d9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.405061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0afd137e-1533-4129-9537-ebc4c058c2f9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:39.406737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0afd137e-1533-4129-9537-ebc4c058c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.455323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.456159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.456511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.456854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.458825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.481409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1130/2218] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:39.490431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882ada-5e02-44b4-acdd-addb867adf49 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.491252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882ada-5e02-44b4-acdd-addb867adf49 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.491719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b882ada-5e02-44b4-acdd-addb867adf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.518563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0b882ada-5e02-44b4-acdd-addb867adf49 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:39.519940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0b882ada-5e02-44b4-acdd-addb867adf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.525490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.527015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e009ff-599d-4dee-ad4a-1be07cfdf436 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.532714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1131/2219] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:39.534287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101b39-9523-41e6-8211-36b3748b96d6 None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.534643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101b39-9523-41e6-8211-36b3748b96d6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.534998 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22101b39-9523-41e6-8211-36b3748b96d6 None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.536534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1132/2220] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:38:39.537183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22101b39-9523-41e6-8211-36b3748b96d6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.537569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1089/2221] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:38:39.551209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22101b39-9523-41e6-8211-36b3748b96d6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.556081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.557398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.557835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.558173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.558939 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:39.559244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.561337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.564937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.564937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.564937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.585867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] MFA Rules not processed for user `3bd6b54079a045dd9a526fafe8de8bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:39.588800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.595354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1133/2222] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:38:39.596035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.627621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777dd6-1674-4fb8-be7a-a7388f586f51 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.648512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.649111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.649398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.649679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.650439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.652885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.723805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1134/2223] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:39.726370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.729067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1090/2224] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:39.802272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3380-8289-4168-aaba-a06680068622 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.808280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.809457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.810204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.810634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.811611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.813739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.844244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.845975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:39.848070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1135/2225] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:39 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:39.851405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.875275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b9467-f8a6-4d21-bbaf-682c48db3054 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.876731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ce63-c88b-47f3-9b81-9a42b302d7e4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.889344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.889590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.889881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.890498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.891006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.895773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.906080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:39.917386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1136/2226] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:38:39.918805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.931066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7848-7c98-4bb5-9cfa-00a88432e7ae admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:39.936911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:39.939093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:39.940658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:39.941453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:39.941960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:39.942899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:39.946433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:39.977153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1137/2227] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:39.980277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:39.998664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f345-9c05-4d9c-95e7-314db661a7be admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:40.002333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:40.003387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:40.004232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:40.004587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:40.011684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:40.013131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:40.013420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:40.013697 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] PATH_INFO: `/v3/projects/daff3a2fbb1745519fa8ea28c2c084c3/users/58e0d069005b46268544e890ce08ee2b/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:40.019916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:40.022777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbff7d1-3a2a-49bb-8033-13c002177593 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:40.045148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1091/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:38:40.047004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daff3a2fbb1745519fa8ea28c2c084c3, user_id=58e0d069005b46268544e890ce08ee2b, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:40.049725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:40.073262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1138/2229] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:39 2022] PUT /identity/v3/projects/daff3a2fbb1745519fa8ea28c2c084c3/users/58e0d069005b46268544e890ce08ee2b/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:40.079060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:40.081947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:40.082244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:40.127467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:40.132002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:40.227926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1092/2230] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:40.237226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:40.239365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:40.312030 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351dfd8-41aa-42e4-bf08-4380c22002b8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:40.315721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:40.316396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:40.316682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:40.316956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:40.326853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:40.328340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:40.338604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1139/2231] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:40.535160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:40.536711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:40.551564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4282e-5ee2-4006-93a6-e7401516467e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:40.563796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:40.565272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:40.565621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:40.565906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] PATH_INFO: `/v3/users/658f3a35ad8046589d2d1f1d3e4b37c0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:40.567530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=658f3a35ad8046589d2d1f1d3e4b37c0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:40.570337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:40.631225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658f3a35ad8046589d2d1f1d3e4b37c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:40.632410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658f3a35ad8046589d2d1f1d3e4b37c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:40.653509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658f3a35ad8046589d2d1f1d3e4b37c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:40.662594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658f3a35ad8046589d2d1f1d3e4b37c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:40.663320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658f3a35ad8046589d2d1f1d3e4b37c0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:40.680841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1093/2232] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:40 2022] DELETE /identity/v3/users/658f3a35ad8046589d2d1f1d3e4b37c0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:41.061464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:41.071554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314535-4c1f-4c9b-971c-f73582e13f38 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:41.153991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:41.155590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:41.155988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:41.156341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] PATH_INFO: `/v3/projects/27ad351e77294268843e41760f697341` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:41.157760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ad351e77294268843e41760f697341)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:41.160731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:41.234082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ad351e77294268843e41760f697341', 'request_id': 'req-82f6aea4-708c-481a-a8a7-d9939d4e3e94'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.240735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ad351e77294268843e41760f697341', 'request_id': 'req-82f6aea4-708c-481a-a8a7-d9939d4e3e94'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.252336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ad351e77294268843e41760f697341', 'request_id': 'req-82f6aea4-708c-481a-a8a7-d9939d4e3e94'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.266597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:41.272959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] The token cache is being invalidate because project 27ad351e77294268843e41760f697341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:41.273603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.274418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.279576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1140/2233] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:41 2022] DELETE /identity/v3/projects/27ad351e77294268843e41760f697341 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:41.281148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:41.282702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:41.315987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc705f-7e24-4990-8eb2-b797baa77fbb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:41.364801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:41.411502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:41.412577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:41.414125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:41.414125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] PATH_INFO: `/v3/users/8d9e9669ed4e416c9b687c3320c2308a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:41.417319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9e9669ed4e416c9b687c3320c2308a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:41.424922 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:41.495828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9e9669ed4e416c9b687c3320c2308a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.496873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9e9669ed4e416c9b687c3320c2308a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.511079 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9e9669ed4e416c9b687c3320c2308a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.524048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9e9669ed4e416c9b687c3320c2308a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.524850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4573869-978a-4d8e-801a-4f24d10710ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9e9669ed4e416c9b687c3320c2308a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.540289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1094/2234] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:41 2022] DELETE /identity/v3/users/8d9e9669ed4e416c9b687c3320c2308a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:41.728021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:41.731266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:41.758794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6aea4-708c-481a-a8a7-d9939d4e3e94 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:41.842860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:41.844064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:41.844822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:41.845102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] PATH_INFO: `/v3/projects/0489aed38d164cac902639029accd34a` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:41.846591 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0489aed38d164cac902639029accd34a)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:41.850282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:41.911820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0489aed38d164cac902639029accd34a', 'request_id': 'req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.918636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0489aed38d164cac902639029accd34a', 'request_id': 'req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.926745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0489aed38d164cac902639029accd34a', 'request_id': 'req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.940789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] The token cache is being invalidate because project 0489aed38d164cac902639029accd34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:41.941001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.941656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9cf9a6f4-7786-4d0f-b904-1522c38a1f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:41.944021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1141/2235] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:41 2022] DELETE /identity/v3/projects/0489aed38d164cac902639029accd34a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:41.948835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:41.949125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:41.949405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:41.969519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:41.970713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:42.052650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1095/2236] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:42.061753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.062320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.062589 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.085448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:42.086963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:42.102244 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1142/2237] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:42.102879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.113536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38382e-63cb-4608-bbc4-711699398408 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.120704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.121745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.121918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.122190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.124397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:42.124774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.126483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.143403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1096/2238] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:38:42.144975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.154945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28913-c487-4a91-951f-08945da0e2f5 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.160081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.161129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.161129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.161431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.162213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.164436 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.194762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1143/2239] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:42.199464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.215641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89f381-bbdc-4494-8a51-0087319b75eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.271552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:42.309162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.310873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.311247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.311624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.312768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.314795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.345058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1097/2240] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:42.345338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.360613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fce3-a67a-4b98-8110-7cee94902c43 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.377571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.378489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.378848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.379260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.380134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.381885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.401284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1144/2241] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:42.405010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.423931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714e06d-f753-4709-9018-bd7ce31fa5cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.435028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.436161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.436644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.436921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.437693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.439307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217c53b-f6a9-4f9c-bb75-ba1f9fa543d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.453751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1098/2242] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:38:42.457570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.469551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9651-98c7-462b-92b2-0ba1743475d7 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.476201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.477629 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.477919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.478200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] PATH_INFO: `/v3/projects/13480b138bba45389d1ba337134a7f1a/users/b55ac1087f09443d9a7bed2f8e81b4bf/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.499275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13480b138bba45389d1ba337134a7f1a, user_id=b55ac1087f09443d9a7bed2f8e81b4bf, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.502851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.519120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1145/2243] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:42 2022] PUT /identity/v3/projects/13480b138bba45389d1ba337134a7f1a/users/b55ac1087f09443d9a7bed2f8e81b4bf/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:42.529840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.530156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.530438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.563937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] MFA Rules not processed for user `b55ac1087f09443d9a7bed2f8e81b4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:42.565660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:42.658635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1099/2244] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:42.663104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.705068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:42.770880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c0bcc-5b5c-4a6d-b000-5b213031109e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.785679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.786232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.786726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.787207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.788035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.790167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.819925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1146/2245] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:42.824216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.840509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4e602-226f-435a-bdca-6eb467ee6319 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:42.923639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:42.924608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:42.925060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:42.925427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:42.926198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:42.928382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:42.971438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1100/2246] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:42 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:42.972353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:42.996211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd458712-1f1e-451a-bafd-2705d835451d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:43.016590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:43.016880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.017164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.017582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.021242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:43.024211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:43.050474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1147/2247] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:43.054877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:43.073572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc7b8e-6f2b-40bd-a5d1-fe3b1533f162 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:43.086644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:43.087573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.088434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.088854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] PATH_INFO: `/v3/projects/ad86d006d9164ea4ac286b94a9621b1e/users/33c7fa835ed7480abc207af7fa483b9b/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.124946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad86d006d9164ea4ac286b94a9621b1e, user_id=33c7fa835ed7480abc207af7fa483b9b, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:43.128161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:43.145324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1101/2248] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:43 2022] PUT /identity/v3/projects/ad86d006d9164ea4ac286b94a9621b1e/users/33c7fa835ed7480abc207af7fa483b9b/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:43.149401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:43.161955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2016d-5954-4295-9d4c-a1b0f7edea97 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:43.258107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:43.258809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.259099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.259381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.261769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:43.263435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91ee8f-123b-4bcc-aa3e-0607bea65b26 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:43.278314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1148/2249] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:43.282314 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:43.294666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2630-e33e-443e-baa3-f30a005ff9c6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:43.300313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:43.301221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.301505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.301802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] PATH_INFO: `/v3/projects/ad86d006d9164ea4ac286b94a9621b1e/users/33c7fa835ed7480abc207af7fa483b9b/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.303860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad86d006d9164ea4ac286b94a9621b1e, user_id=33c7fa835ed7480abc207af7fa483b9b, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:43.306605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:43.310711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:43.325196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1102/2250] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:43 2022] PUT /identity/v3/projects/ad86d006d9164ea4ac286b94a9621b1e/users/33c7fa835ed7480abc207af7fa483b9b/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:43.341652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593098-d06b-466f-8627-afba0c1d460e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.343357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593098-d06b-466f-8627-afba0c1d460e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.343642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593098-d06b-466f-8627-afba0c1d460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.409946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ee593098-d06b-466f-8627-afba0c1d460e None None] MFA Rules not processed for user `33c7fa835ed7480abc207af7fa483b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:43.411255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ee593098-d06b-466f-8627-afba0c1d460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:43.653481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1149/2251] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:43.665321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:43.667934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:43.813620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083c85-727a-404a-b925-00611d452295 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:43.817449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:43.818124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:43.818616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:43.818922 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:43.897059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:43.898584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:43.911024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1103/2252] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:45.776861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:45.777448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:45.777448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:45.809639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] MFA Rules not processed for user `58e0d069005b46268544e890ce08ee2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:45.811075 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:45.820747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:45.889376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1150/2253] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:45.906283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:45.907818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:46.003588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84babc5-e998-4540-95b6-27b94c85c0f6 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:46.009453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:46.010163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:46.010442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:46.010737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:46.019422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:46.021237 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a03caa-2fb4-4a5a-93a2-a4972d3cb5c3 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:46.036538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1104/2254] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:46.129003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:46.132734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:46.298053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22643d9d-92f4-4ad0-a0ec-aff33fc7f259 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:46.303698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:46.305595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:46.306034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:46.306545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] PATH_INFO: `/v3/users/84159744669a4f328f860fbc45822969` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:46.313237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=84159744669a4f328f860fbc45822969)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:46.317705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:46.439288 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84159744669a4f328f860fbc45822969'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:46.477546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84159744669a4f328f860fbc45822969'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:46.478564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84159744669a4f328f860fbc45822969'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:46.482761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84159744669a4f328f860fbc45822969'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:46.540064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84159744669a4f328f860fbc45822969'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:46.572074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1151/2255] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:46 2022] DELETE /identity/v3/users/84159744669a4f328f860fbc45822969 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:46.575001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:46.575535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:46.576047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:46.642981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:46.645883 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:46.763741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:46.833259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1105/2256] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:46.845208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:46.846800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:46.924286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:46.980473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e79f-63ce-4e9f-8949-bc34df978c6e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:46.984370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:46.985785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:46.986163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:46.997956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:47.010308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:47.013191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:47.034946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1152/2257] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:47.745749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:47.751964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:47.772905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dc3b8-7270-496e-be0b-55d2797d96b0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:47.780512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:47.781587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:47.782216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:47.782654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] PATH_INFO: `/v3/projects/68560c74ae4b428f90be95b64ca55c3f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:47.784954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=68560c74ae4b428f90be95b64ca55c3f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:47.787136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:47.885514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68560c74ae4b428f90be95b64ca55c3f', 'request_id': 'req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:47.894315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68560c74ae4b428f90be95b64ca55c3f', 'request_id': 'req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:47.901604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68560c74ae4b428f90be95b64ca55c3f', 'request_id': 'req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:47.927541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] The token cache is being invalidate because project 68560c74ae4b428f90be95b64ca55c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:47.928029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:47.928745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-db8d4ffa-5fd7-4dc5-9c94-7aad6a634bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:47.930480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:47.932278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1106/2258] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:47 2022] DELETE /identity/v3/projects/68560c74ae4b428f90be95b64ca55c3f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:47.933405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:47.938601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860402f-90a9-406f-8f34-cb40fb4e7d5c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:47.939746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860402f-90a9-406f-8f34-cb40fb4e7d5c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:47.940367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860402f-90a9-406f-8f34-cb40fb4e7d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:47.947511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c426d-5b13-4c4a-8b0b-85a126252d82 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:47.970184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e860402f-90a9-406f-8f34-cb40fb4e7d5c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:47.971791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e860402f-90a9-406f-8f34-cb40fb4e7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.020000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.020906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.021192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.021472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] PATH_INFO: `/v3/users/b55ac1087f09443d9a7bed2f8e81b4bf` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.022719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55ac1087f09443d9a7bed2f8e81b4bf)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.025020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.069009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1107/2259] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:48.070766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.071133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.071641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.087329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55ac1087f09443d9a7bed2f8e81b4bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:48.091340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:48.092989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.101312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1108/2260] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:48.106467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.119817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68915c-46ae-42f7-8481-7da390b4c39c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.122469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55ac1087f09443d9a7bed2f8e81b4bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:48.123292 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55ac1087f09443d9a7bed2f8e81b4bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:48.124000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55ac1087f09443d9a7bed2f8e81b4bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:48.125864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.126618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.126977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.127347 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.128195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:48.128574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.130644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.133688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e4e64d5a-dde0-4592-ba82-5fd97107d248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55ac1087f09443d9a7bed2f8e81b4bf'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:48.154753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1153/2261] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:47 2022] DELETE /identity/v3/users/b55ac1087f09443d9a7bed2f8e81b4bf => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:48.154948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1109/2262] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:38:48.157242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.158673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.158972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.159284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.160767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.179950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230800c-0031-4395-9f7f-69aaaae9d9ec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.182239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:48.183867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.240899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:48.257692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:48.279726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1154/2263] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:48.290453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.293322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.293821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.294020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.294308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.294588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.295294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.297330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.336954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1110/2264] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:48.338384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.393824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d353aca-d0fb-41cf-9465-552d28634d3b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.449146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03aa5-0082-4a8d-b137-0d22fd562198 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.457727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.458574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.458860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.459143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.459880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.462016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.467133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.468531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.469063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.469464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.490538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.492550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.495899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1111/2265] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:48.501181 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.510674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1155/2266] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:38:48.528463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a2877-c8a2-480c-ae45-8a1681142bd4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.534738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.535430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.536032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.536136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.536885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.538510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.554667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1112/2267] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:38:48.559474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.572060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca41e1e-b560-4410-91e9-b2ccba575425 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.582859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.583641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.583963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.584286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.585636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.588975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8149cc2-2120-4a78-b06d-6b2354daeca6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.605307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1156/2268] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:48.605934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.622036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe0d58-5f5c-4e3b-88b7-01108f2c819b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.645366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.646527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.646916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.647395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] PATH_INFO: `/v3/projects/1588191e22ee40979999efc6dfc826bf/users/5b79be1e1cd04b7695373fae08d97039/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.688092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1588191e22ee40979999efc6dfc826bf, user_id=5b79be1e1cd04b7695373fae08d97039, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.690262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364c4f4-eaa3-4f82-901f-66fa163c3508 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.711325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1113/2269] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:48 2022] PUT /identity/v3/projects/1588191e22ee40979999efc6dfc826bf/users/5b79be1e1cd04b7695373fae08d97039/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:48.717364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.717654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.717994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.742868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:48.745278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.835681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.836290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.836290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.868041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] MFA Rules not processed for user `58e0d069005b46268544e890ce08ee2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:48.869385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.870303 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1157/2270] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:48.881569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.883372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:49.026685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1114/2271] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:49.051555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:49.083388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdc1a5-cc6f-4a9b-9564-8cd0b4dbb960 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:49.090907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:49.091522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:49.091837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:49.092154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:49.102729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:49.104364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:49.115884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1158/2272] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:49.285191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:49.286152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:49.312072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf8ade-8443-479c-80b8-e2876e11effa None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:49.324873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:49.326295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:49.326684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:49.326840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] PATH_INFO: `/v3/projects/13480b138bba45389d1ba337134a7f1a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:49.330569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13480b138bba45389d1ba337134a7f1a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:49.334502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:49.418630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:49.484514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13480b138bba45389d1ba337134a7f1a', 'request_id': 'req-8fa81c0c-bf6f-4977-bbaf-0821592a132a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.511521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13480b138bba45389d1ba337134a7f1a', 'request_id': 'req-8fa81c0c-bf6f-4977-bbaf-0821592a132a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.521341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13480b138bba45389d1ba337134a7f1a', 'request_id': 'req-8fa81c0c-bf6f-4977-bbaf-0821592a132a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.555266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] The token cache is being invalidate because project 13480b138bba45389d1ba337134a7f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:49.555566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.558565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.562625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1115/2273] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:49 2022] DELETE /identity/v3/projects/13480b138bba45389d1ba337134a7f1a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:49.567118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:49.589788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60699093-0257-475e-becd-f888b4400bbd service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:49.739522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:49.741339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:49.741807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:49.742803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] PATH_INFO: `/v3/users/33c7fa835ed7480abc207af7fa483b9b` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:49.745185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c7fa835ed7480abc207af7fa483b9b)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:49.752415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:49.892540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c7fa835ed7480abc207af7fa483b9b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.921236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c7fa835ed7480abc207af7fa483b9b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.922576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c7fa835ed7480abc207af7fa483b9b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.923962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c7fa835ed7480abc207af7fa483b9b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.943146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4689e65e-8926-4afb-bb16-a4331cfbf438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c7fa835ed7480abc207af7fa483b9b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:49.976679 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1159/2274] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:49 2022] DELETE /identity/v3/users/33c7fa835ed7480abc207af7fa483b9b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:51.090662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:51.091836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:51.136392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa81c0c-bf6f-4977-bbaf-0821592a132a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:51.348037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:51.349158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:51.349541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:51.349931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] PATH_INFO: `/v3/projects/ad86d006d9164ea4ac286b94a9621b1e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:51.358187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad86d006d9164ea4ac286b94a9621b1e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:51.360317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:51.497418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad86d006d9164ea4ac286b94a9621b1e', 'request_id': 'req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:51.512534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad86d006d9164ea4ac286b94a9621b1e', 'request_id': 'req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:51.527964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad86d006d9164ea4ac286b94a9621b1e', 'request_id': 'req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:51.569150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] The token cache is being invalidate because project ad86d006d9164ea4ac286b94a9621b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:51.569583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:51.571253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a98254d-f7bc-4ee9-aa4e-bd3bb76cc7fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:51.574413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1116/2275] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:51 2022] DELETE /identity/v3/projects/ad86d006d9164ea4ac286b94a9621b1e => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:51.579531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:51.580067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:51.580161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:51.627560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:51.628834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:51.763943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1160/2276] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:51.772838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:51.773589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:51.774082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:51.829896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:51.831431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:51.847509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1117/2277] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:51.850988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:51.857104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:51.867876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb83d68-b3f2-4e79-8e71-4f816760f325 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:51.890972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:51.893514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:51.896162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:51.898945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:51.900234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:38:51.901266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:51.904760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:51.935496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1161/2278] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:38:51.940020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:51.953577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf9ae-18d8-40ff-8802-126ae0614a44 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:51.972582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:51.973793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:51.974134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:51.974473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:51.975738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:51.984296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.059165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1118/2279] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:51 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:38:52.060711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:52.083529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17fca4-51b1-4b27-94d5-cf519b71d02e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:52.168145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:52.169018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.169610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.170036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.171397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:52.174395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.228636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1162/2280] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:52 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:52.232315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:52.244771 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687e498-f453-4007-9959-0976c2d764f9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:52.260978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:52.262322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.263551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.265554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.267486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:52.270006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.276279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:52.300792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1119/2281] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:52.304975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:52.317109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574f804-a4a6-42e1-bc0a-a7913314cb3a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:52.325748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:52.327643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.328074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.329956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.331127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:52.333086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f87783-d0b4-459b-b3e2-594ccec79173 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.372619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:52.373175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1163/2282] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:52.383664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6e2f-6d42-448e-872d-ed340c044812 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:52.407931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:52.412034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.414283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.415227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] PATH_INFO: `/v3/projects/d154d333ad7947a78489ba93525b1679/users/fc69b43aa6e54ba7a12513b721ebc8b5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.457931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d154d333ad7947a78489ba93525b1679, user_id=fc69b43aa6e54ba7a12513b721ebc8b5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:52.465482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.489944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1120/2283] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:52 2022] PUT /identity/v3/projects/d154d333ad7947a78489ba93525b1679/users/fc69b43aa6e54ba7a12513b721ebc8b5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:38:52.500769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eb7f-2bfe-488d-9583-75142f486ddf None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.501061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eb7f-2bfe-488d-9583-75142f486ddf None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.501348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eb7f-2bfe-488d-9583-75142f486ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.531047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8822eb7f-2bfe-488d-9583-75142f486ddf None None] MFA Rules not processed for user `fc69b43aa6e54ba7a12513b721ebc8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:52.532309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8822eb7f-2bfe-488d-9583-75142f486ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:52.665739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1164/2284] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:52.676699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:52.687349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:52.890454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47ba4d-6f20-4de9-8f75-2c25dfc7c080 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:52.902900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:52.903833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:52.904126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:52.904416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:52.925828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:52.927496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:52.954618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1121/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:38:54.770023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000f40-20cf-40b1-8218-7dc2f721ee8e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:54.771056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000f40-20cf-40b1-8218-7dc2f721ee8e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:54.771644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b000f40-20cf-40b1-8218-7dc2f721ee8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:54.808002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0b000f40-20cf-40b1-8218-7dc2f721ee8e None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:54.809372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0b000f40-20cf-40b1-8218-7dc2f721ee8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:54.998232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1165/2286] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:55.091251 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:55.096216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:55.219106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0eeb8-f54b-4f72-a300-784ba28da469 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:55.227747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:55.228716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:55.229146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:55.229654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:55.244939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:55.246838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:55.261560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1122/2287] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:38:57.172908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53c67d-7345-494d-b25f-b47cb81f2ac5 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.173454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53c67d-7345-494d-b25f-b47cb81f2ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.173454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53c67d-7345-494d-b25f-b47cb81f2ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.183731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:57.236712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0c53c67d-7345-494d-b25f-b47cb81f2ac5 None None] MFA Rules not processed for user `5b79be1e1cd04b7695373fae08d97039`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:57.238364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0c53c67d-7345-494d-b25f-b47cb81f2ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:57.325770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1166/2288] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:38:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:57.330829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.332346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:57.381027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.381509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.382011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.406098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:57.407628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:57.416114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1167/2289] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:57.435952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.436658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9120b-1c43-45f3-ae38-3bc4a2351856 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.441354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.442270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.443599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.444760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.445604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.448553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.449551 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43b51b-c4c9-4047-a34d-2b346033c553 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.461059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:57.466490 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.468260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.468654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.469000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.480713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.482354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.494781 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1168/2290] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:38:57.497042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1123/2291] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:38:57 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:57.511620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.529868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324aee6-53e6-4f2b-8d87-a19faf1d00fa service swift] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.629083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.629929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.630253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.630594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.631354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.635563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.664755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1169/2292] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:38:57 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:38:57.666261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.677595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f0b08-2dd2-4e4d-aec9-9393c2000f9e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.683182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.683980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.684331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.684694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.685493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.687214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.701303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1124/2293] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:38:57.705293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.717389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52911547-00ae-4ae7-be45-7ecbeaead54c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.722653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.723855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.724168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.724454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] PATH_INFO: `/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd/users/8f1be900789c47daa9d7cd32205177a5/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.741955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d395cff7746cdbfef7af5ac421ccd, user_id=8f1be900789c47daa9d7cd32205177a5, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.745354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e460f8a-e494-49c6-ace7-2ad94c1b6177 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.765917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1170/2294] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:57 2022] PUT /identity/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd/users/8f1be900789c47daa9d7cd32205177a5/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:57.769834 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.772231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.772524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.772806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.785447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cb871-7a0c-450b-9710-45b61095f2e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.791613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:57.792900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:57.835210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:57.857681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.858442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.858842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.859131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.860011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.861640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.874949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1125/2295] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:38:57.880561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1171/2296] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:57.880894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.892758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:57.893034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25090e42-f0a9-4f77-8271-d1334caa26da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:57.898747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:57.900072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:57.900072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:57.900293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] PATH_INFO: `/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd/users/8f1be900789c47daa9d7cd32205177a5/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:57.903404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9d395cff7746cdbfef7af5ac421ccd, user_id=8f1be900789c47daa9d7cd32205177a5, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:57.907295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb473d-83ab-4394-9141-9af7d622cf18 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:57.927960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1126/2297] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:38:57 2022] PUT /identity/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd/users/8f1be900789c47daa9d7cd32205177a5/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:57.980294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3201-27dd-469f-9286-f44c2e011d79 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:58.057229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:58.057883 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:58.058161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:58.058434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:58.150728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:58.150728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:58.175092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1172/2298] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:38:58.619596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c01baf-c1db-466e-8fd5-9634e2b3766b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:58.620119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c01baf-c1db-466e-8fd5-9634e2b3766b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:58.620285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c01baf-c1db-466e-8fd5-9634e2b3766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:58.639879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e5c01baf-c1db-466e-8fd5-9634e2b3766b None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:58.641077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5c01baf-c1db-466e-8fd5-9634e2b3766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:58.648984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1127/2299] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:38:58.665222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:58.677188 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c565-7786-43f9-b910-126f6a1c262b service swift] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:58.683086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:58.683823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:58.684091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:58.684368 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:58.695076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:58.696761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:58.706607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1173/2300] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:39:03.530919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df17702-8504-4501-84fa-d898cd0fe95a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:03.531797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df17702-8504-4501-84fa-d898cd0fe95a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:03.532083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df17702-8504-4501-84fa-d898cd0fe95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:03.559573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6df17702-8504-4501-84fa-d898cd0fe95a None None] MFA Rules not processed for user `8f1be900789c47daa9d7cd32205177a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:03.560884 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6df17702-8504-4501-84fa-d898cd0fe95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:03.633388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1128/2301] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:03.648068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:03.650945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:03.667790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:03.742151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37786-169c-4599-959b-784f3ebdee6f service swift] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:03.749532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:03.750199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:03.750480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:03.750752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:03.828724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:03.830340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:03.840710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1174/2302] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:05.090602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0224cd-6b25-484a-a5ca-4c6d092f8bcf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:05.091634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0224cd-6b25-484a-a5ca-4c6d092f8bcf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:05.092169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0224cd-6b25-484a-a5ca-4c6d092f8bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:05.114054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2c0224cd-6b25-484a-a5ca-4c6d092f8bcf None None] MFA Rules not processed for user `5b79be1e1cd04b7695373fae08d97039`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:05.115341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2c0224cd-6b25-484a-a5ca-4c6d092f8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:05.127812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1129/2303] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:05.612064 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:05.614219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:05.696195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44c6b8-07a3-402d-a732-6e555478f827 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:05.701198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:05.702193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:05.702394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:05.702673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] PATH_INFO: `/v3/users/8fd0d94821f640a1949424c64dd71b65` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:05.703884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd0d94821f640a1949424c64dd71b65)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:05.705845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:05.765509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd0d94821f640a1949424c64dd71b65'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:05.770632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:05.782624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd0d94821f640a1949424c64dd71b65'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:05.783550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd0d94821f640a1949424c64dd71b65'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:05.784232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd0d94821f640a1949424c64dd71b65'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:05.792625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd0d94821f640a1949424c64dd71b65'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:05.807447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1175/2304] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:05 2022] DELETE /identity/v3/users/8fd0d94821f640a1949424c64dd71b65 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:05.811875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:05.812265 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:05.812453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:05.831569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:05.832891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:05.884645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:05.919842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1130/2305] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:05.930213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:05.932559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:06.023147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ddb9d-9c7f-4bc6-8f42-6bf803eeadb2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.027737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.029107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.029393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.029680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.039700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.041721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93cb22-b2a7-4296-abac-1f46921dde9b service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.052855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1176/2306] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:06.315053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.317535 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:06.336974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355ce0a-79e3-406b-9544-bcb550c66eda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.342957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.343932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.344212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.344486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] PATH_INFO: `/v3/projects/9e6cd40fc3354dfcb3821979cd6c9889` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.346848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6cd40fc3354dfcb3821979cd6c9889)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.349946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.405355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6cd40fc3354dfcb3821979cd6c9889', 'request_id': 'req-267bd3be-8ec7-4248-b390-49b96a826d0b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:06.411330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6cd40fc3354dfcb3821979cd6c9889', 'request_id': 'req-267bd3be-8ec7-4248-b390-49b96a826d0b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:06.417878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6cd40fc3354dfcb3821979cd6c9889', 'request_id': 'req-267bd3be-8ec7-4248-b390-49b96a826d0b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:06.438192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] The token cache is being invalidate because project 9e6cd40fc3354dfcb3821979cd6c9889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:06.438513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:06.439135 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-267bd3be-8ec7-4248-b390-49b96a826d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:06.440721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1131/2307] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:06 2022] DELETE /identity/v3/projects/9e6cd40fc3354dfcb3821979cd6c9889 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:06.445873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.446279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.447154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.465227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:06.466350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:06.529515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1177/2308] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:06.537151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95117c-6c71-428f-9758-cbface806411 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.537434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95117c-6c71-428f-9758-cbface806411 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.537709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95117c-6c71-428f-9758-cbface806411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.556646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f95117c-6c71-428f-9758-cbface806411 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:06.558137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f95117c-6c71-428f-9758-cbface806411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:06.566629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1132/2309] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:06.570910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.582409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ff08b-965d-4b76-8cf6-57d28ae09d62 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.587100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.588412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.588691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.588961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.589680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:06.589981 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.591559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.604224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1178/2310] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:06.608293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95117c-6c71-428f-9758-cbface806411 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.618446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95117c-6c71-428f-9758-cbface806411 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.625668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.626387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.626672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.626949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.627638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.629565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.674676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1133/2311] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:06 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:06.679073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.692940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02a83d-ab36-41c8-966c-021fb22ad4ed admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.765120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.765906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.766192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.766457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.767151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.769095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.777187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:06.818156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1179/2312] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:06 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:06.820011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.841820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cd655-dcfa-48f2-aa95-4eb9b29e15d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.847039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.847669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.852183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.852724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.855121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.857123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.869020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1134/2313] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:39:06.873307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.895565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d82f0-f7f6-4dab-b898-407d76c448ac admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.901914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.902909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.903264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.903809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.904827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.906725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf690e0-3264-429a-b15e-7e73a8f11e89 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:06.925674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1180/2314] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:06.929899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:06.950432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f07df-f563-40a2-af17-9f9c3b5a4c54 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:06.955505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:06.956603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:06.957022 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:06.957418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] PATH_INFO: `/v3/projects/bf34c09020ce4f75b07bdfc37c77d82f/users/bece6c37b7484f0c8ac24cb74a6e6200/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:06.988521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf34c09020ce4f75b07bdfc37c77d82f, user_id=bece6c37b7484f0c8ac24cb74a6e6200, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:06.991476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:07.026478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1135/2315] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:06 2022] PUT /identity/v3/projects/bf34c09020ce4f75b07bdfc37c77d82f/users/bece6c37b7484f0c8ac24cb74a6e6200/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:07.032119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11aa70-1db5-4992-ab5a-6f771098f32d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:07.032746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11aa70-1db5-4992-ab5a-6f771098f32d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:07.033231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11aa70-1db5-4992-ab5a-6f771098f32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:07.058214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ea11aa70-1db5-4992-ab5a-6f771098f32d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:07.059719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ea11aa70-1db5-4992-ab5a-6f771098f32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:07.166089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1181/2316] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:07.178020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:07.180003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:07.271702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e6cf-26c0-4564-a431-6568050f2a15 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:07.276603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:07.277562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:07.277947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:07.278606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:07.293127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:07.294752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:07.299213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:07.309194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1136/2317] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:12.137673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.138230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.138230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.169094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] MFA Rules not processed for user `bece6c37b7484f0c8ac24cb74a6e6200`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:12.170444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:12.268667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1182/2318] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:12.273271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.284703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb11b61-883f-4de4-a2e8-f58ecfeea7d4 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.289975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.291221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.291540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.291796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.292496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.294473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.324279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1137/2319] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:12.328321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.340205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7a680-26d4-4352-b63a-16e0c300edff None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.421449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.422150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.422430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.422702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.423404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.427800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.446106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:12.456149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1183/2320] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:12.459232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.470857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980d5d9-58a7-47a9-b756-b23985693616 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.476433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.477226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.477525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.478052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.478858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.480625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.501597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1138/2321] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:12.506172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.523595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09348f-60a0-4d6b-87bb-b245cfb4ae39 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.538146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.539224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.539643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.540278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] PATH_INFO: `/v3/projects/f2525b377f5c4128ba15bc7bfebca951/users/a25ecb25c98044b6b438098022ee4fcc/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.557399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2525b377f5c4128ba15bc7bfebca951, user_id=a25ecb25c98044b6b438098022ee4fcc, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.560294 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.579728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1184/2322] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:12 2022] PUT /identity/v3/projects/f2525b377f5c4128ba15bc7bfebca951/users/a25ecb25c98044b6b438098022ee4fcc/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:12.583981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.595414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da02e7-9388-4de7-a791-a72a5610fc2a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.662170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.662761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.663048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.663323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.664242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.666453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53942f-4571-427a-951a-506f38298ba5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.678939 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1139/2323] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:39:12.683898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.694728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0293a27-f5ca-4eee-b2c4-17fdac15b074 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.700046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.700961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.701225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.701496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] PATH_INFO: `/v3/projects/f2525b377f5c4128ba15bc7bfebca951/users/a25ecb25c98044b6b438098022ee4fcc/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.703586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2525b377f5c4128ba15bc7bfebca951, user_id=a25ecb25c98044b6b438098022ee4fcc, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.706321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.718212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1185/2324] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:12 2022] PUT /identity/v3/projects/f2525b377f5c4128ba15bc7bfebca951/users/a25ecb25c98044b6b438098022ee4fcc/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:39:17.956610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd065-09be-4c35-885e-2798dc3c8a24 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.957475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd065-09be-4c35-885e-2798dc3c8a24 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.957908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd065-09be-4c35-885e-2798dc3c8a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.999428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ccacd065-09be-4c35-885e-2798dc3c8a24 None None] MFA Rules not processed for user `a25ecb25c98044b6b438098022ee4fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:18.002081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ccacd065-09be-4c35-885e-2798dc3c8a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:18.131329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1140/2325] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:20.647263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:20.649021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:20.725418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb7f73-8cec-4a4a-ab38-79e9e43485e5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:20.729705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:20.730370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:20.730656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:20.730936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:20.811925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:20.813385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:20.842287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1186/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:20.950218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60443d-45d9-437a-8695-402c3affa1eb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:20.950716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60443d-45d9-437a-8695-402c3affa1eb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:20.950886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60443d-45d9-437a-8695-402c3affa1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:20.970115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d60443d-45d9-437a-8695-402c3affa1eb None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:20.971476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d60443d-45d9-437a-8695-402c3affa1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:21.030197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:21.047305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1141/2327] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:21.061586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:21.062685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:21.151842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94bc68-cdd3-429d-a2ff-4c7d02d20215 service glance] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:21.157078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:21.157748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:21.158037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:21.158297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:21.167694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:21.169419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:21.179532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1187/2328] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:39:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:39:30.357207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:30.357740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:30.357740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:30.375274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] MFA Rules not processed for user `bece6c37b7484f0c8ac24cb74a6e6200`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:30.376495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:30.383423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1142/2329] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:31.176276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:40.369760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:43.339871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:43.341315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:43.427148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155d225-a8ed-426f-98bd-393dcac312c3 service swift] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:43.439280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:43.440028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.440383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.440728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.444955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:43.460055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:43.462394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82871fc-8203-4284-a51b-dc9ba18d6f59 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:43.473154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1188/2330] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:44.547185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.549426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:44.623336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdcf08-d592-46b5-b459-1f8105df5f8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.628285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.629155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.629440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.629718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] PATH_INFO: `/v3/users/4a1b9c8c4b95435793d501eea0c8580c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.630917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a1b9c8c4b95435793d501eea0c8580c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.632896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.683420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a1b9c8c4b95435793d501eea0c8580c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:44.684107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a1b9c8c4b95435793d501eea0c8580c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:44.697339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a1b9c8c4b95435793d501eea0c8580c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:44.705056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a1b9c8c4b95435793d501eea0c8580c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:44.705782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a1b9c8c4b95435793d501eea0c8580c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:44.720011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1143/2331] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:44 2022] DELETE /identity/v3/users/4a1b9c8c4b95435793d501eea0c8580c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:44.724752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.725340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.725340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.742357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:44.743604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:44.803262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1189/2332] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:44.813168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.814669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:44.875046 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aae13-388e-49ab-a042-ad9d7270ec4c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.878666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.879316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.879624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.879934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.889949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.891455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe573a9a-f185-41df-bb99-bd7dee1369e9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.905967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1144/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:39:45.348788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.350274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.365065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02700f-5160-48f2-b538-190a9f8e3e8c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.376641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.378078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.378521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.378938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] PATH_INFO: `/v3/projects/9a471a0dcde9474cb908566dc35969ca` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.380449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a471a0dcde9474cb908566dc35969ca)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.382506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.469802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a471a0dcde9474cb908566dc35969ca', 'request_id': 'req-a70cefa1-37e6-42fd-be64-748c4003ecc6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:45.476683 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a471a0dcde9474cb908566dc35969ca', 'request_id': 'req-a70cefa1-37e6-42fd-be64-748c4003ecc6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:45.486036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a471a0dcde9474cb908566dc35969ca', 'request_id': 'req-a70cefa1-37e6-42fd-be64-748c4003ecc6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:45.494691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.494972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.495250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.502077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] The token cache is being invalidate because project 9a471a0dcde9474cb908566dc35969ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:45.502520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:45.503315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a70cefa1-37e6-42fd-be64-748c4003ecc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:45.505045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1190/2334] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:45 2022] DELETE /identity/v3/projects/9a471a0dcde9474cb908566dc35969ca => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:45.511207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1faf-8bf8-492f-8e46-7e359474e468 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.511641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1faf-8bf8-492f-8e46-7e359474e468 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.512057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1faf-8bf8-492f-8e46-7e359474e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.517394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:45.518930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.536278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b56e1faf-8bf8-492f-8e46-7e359474e468 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:45.537682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b56e1faf-8bf8-492f-8e46-7e359474e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.539652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:45.548811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:45.598144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1145/2335] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:45.607165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.609369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.624444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1191/2336] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:45.632054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.632754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.633835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.656139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:45.657346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.673123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1192/2337] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:45.677867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.692573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc1a5-9ba0-4e58-920f-538ce35fc4ec None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.698048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.698874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.699163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.699468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.700508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:45.700898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.702576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.716249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1193/2338] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:45.720432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.727327 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3938f5c-7bcb-43f8-8c0f-cf6dedf8df10 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.731457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15563b7c-6321-4a45-887b-47b552bd6eb5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.735093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.736202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.736696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.737052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.738583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.739261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.739537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.739846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.740543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.743598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.748702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.750481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.761925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1146/2339] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:39:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:39:45.779570 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1194/2340] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:45.783890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.834037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468fed6-6037-4771-a5bf-a240dc54a0bb service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.956207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.957328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.958376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.958928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.960697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.963517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.989980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1147/2341] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/users => generated 278 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:45.996386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.018952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca97b-faf2-4271-9202-9b696b76356a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.025890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.027946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.028329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.028662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.029382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.031024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.058087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1195/2342] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:46.060497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.111036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929af63-696e-4479-946a-df55a1c91cb3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.131648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.134420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.135229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.135829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.137241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.143930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e3aa-536a-4b14-8ba8-9afcb1c8f494 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.178407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1148/2343] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:46.183039 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.212476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb815fe-57f2-40cf-b6e8-5a864f34257e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.219673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.220815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.221351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.221728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] PATH_INFO: `/v3/projects/e2636303d07a42e39ac4b2521e3d6d48/users/31ec3bfa694545ce9e4595de1171a66b/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.251905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2636303d07a42e39ac4b2521e3d6d48, user_id=31ec3bfa694545ce9e4595de1171a66b, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.255003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.294905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1196/2344] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:46 2022] PUT /identity/v3/projects/e2636303d07a42e39ac4b2521e3d6d48/users/31ec3bfa694545ce9e4595de1171a66b/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:46.311448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.311745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.312056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.347345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] MFA Rules not processed for user `31ec3bfa694545ce9e4595de1171a66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:46.348689 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.387846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:46.453350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1149/2345] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:46.506098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.508590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.580446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef67cc5-133d-4999-a0ac-d852a295d82f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.584209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.584966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.585351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.585742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.595663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.597268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721659-c335-4eaf-8dc4-b6f6ed6148bb service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.610201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1197/2346] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:46.624887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.692613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac1f19-99f9-4f7e-a144-ee2dc4e56d1a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.698222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.699220 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.699519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.699899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] PATH_INFO: `/v3/users/31ec3bfa694545ce9e4595de1171a66b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.701344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ec3bfa694545ce9e4595de1171a66b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.703475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.763378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ec3bfa694545ce9e4595de1171a66b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:46.764538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ec3bfa694545ce9e4595de1171a66b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:46.778889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ec3bfa694545ce9e4595de1171a66b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:46.788186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ec3bfa694545ce9e4595de1171a66b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:46.789085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ec3bfa694545ce9e4595de1171a66b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:46.807451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1150/2347] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:46 2022] DELETE /identity/v3/users/31ec3bfa694545ce9e4595de1171a66b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:46.812177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.812690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.813092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.833518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:46.834709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.869785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:46.898743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1198/2348] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:46.908616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.910256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.982348 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:46.994566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a18aa-136f-41f4-a28d-93b18dd722a7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.998839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.999491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.999782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.000065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.010372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.011839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c5910-696b-4cb2-8869-e0aeed1212f6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.021796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1151/2349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:47.346585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.349080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.366751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7378728-f755-4063-aa87-cc0dfcd8c77c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.371750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.372901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.373182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.373460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] PATH_INFO: `/v3/projects/e2636303d07a42e39ac4b2521e3d6d48` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.374788 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2636303d07a42e39ac4b2521e3d6d48)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.376914 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.429428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2636303d07a42e39ac4b2521e3d6d48', 'request_id': 'req-efd41ff8-9328-4d79-be53-fea0f095514b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.435208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2636303d07a42e39ac4b2521e3d6d48', 'request_id': 'req-efd41ff8-9328-4d79-be53-fea0f095514b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.442719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2636303d07a42e39ac4b2521e3d6d48', 'request_id': 'req-efd41ff8-9328-4d79-be53-fea0f095514b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.454496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] The token cache is being invalidate because project e2636303d07a42e39ac4b2521e3d6d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:47.454875 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.455597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-efd41ff8-9328-4d79-be53-fea0f095514b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.460564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1199/2350] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:47 2022] DELETE /identity/v3/projects/e2636303d07a42e39ac4b2521e3d6d48 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:47.462863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.463287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.463690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.485416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:47.486713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.574977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1152/2351] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:47.583423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.583712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.584022 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.606479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:47.620941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.638417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1200/2352] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:47.643010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.643837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.645280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.654493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a33ee-1d6b-4ab4-8f35-04899a5e4346 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.660290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.661443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.661928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.662365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.663446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:47.663950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.666403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.671283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e65a0-57a8-40e0-8443-36b8cbe9e5fe None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.677014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.678708 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.678999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.679277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] PATH_INFO: `/v3/users/fc69b43aa6e54ba7a12513b721ebc8b5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.685073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1153/2353] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:47.691577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.692467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc69b43aa6e54ba7a12513b721ebc8b5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.694521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.705630 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659cb37-5f10-429b-9176-510fd7e9adaf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.711109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.711870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.712157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.712434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.713173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.715146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.748809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1154/2354] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:47 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:47.761145 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc69b43aa6e54ba7a12513b721ebc8b5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.761380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.790579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc69b43aa6e54ba7a12513b721ebc8b5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.791256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc69b43aa6e54ba7a12513b721ebc8b5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.791912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc69b43aa6e54ba7a12513b721ebc8b5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.799869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4f52ab5c-5402-4f22-87fe-75e80c3a8854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc69b43aa6e54ba7a12513b721ebc8b5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:47.814852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1201/2355] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:47 2022] DELETE /identity/v3/users/fc69b43aa6e54ba7a12513b721ebc8b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:47.819245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.819540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.819865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.836518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993d30e-8296-4309-82b6-790814bb1ed4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.838014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:47.839355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.871556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:47.903042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.903916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.904288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.904580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.905275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.907196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.914053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:47.919212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1202/2356] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:47.929620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.932221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.941140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1155/2357] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:47 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:47.945126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.948173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:47.965062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b4adb-a328-42b2-8875-2b7663b73a7a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:47.969858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:47.970532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:47.970812 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:47.971083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.971880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.973691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.983958 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1156/2358] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:47.988085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:47.998393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737b21-3712-4522-8b4c-e3f9d36b608a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:48.002997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:48.003695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.004013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.004286 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.005031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:48.006620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:48.013358 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c5b69-94ef-4bee-9ebd-606cba1711e7 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:48.017261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:48.020442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.020951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.021454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.022189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1157/2359] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:39:48.026162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:48.037381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb9c72-ec87-41a7-819e-e2033a53a6ad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:48.042349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:48.043313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.043677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.044048 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] PATH_INFO: `/v3/projects/90731b2dc5a4462184d4baa1de00ebcf/users/6f774d316a224f30b17e1985a51e8198/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.048839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:48.050367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980158f-6f64-4fd0-a635-c3676aa0e2cd service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:48.061218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1203/2360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:39:48.063564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90731b2dc5a4462184d4baa1de00ebcf, user_id=6f774d316a224f30b17e1985a51e8198, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:48.066371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:48.081647 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1158/2361] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:48 2022] PUT /identity/v3/projects/90731b2dc5a4462184d4baa1de00ebcf/users/6f774d316a224f30b17e1985a51e8198/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:39:48.086964 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.087354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.087695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.110290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:48.111585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:48.215601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1204/2362] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:48.230620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:48.236575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:48.317277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dd477-c2a2-4a6a-a9bf-afc4fb10d4c4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:48.322890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:48.324036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.324036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.324280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.335934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:48.338328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8c62-9577-47f5-9534-6aea2556d37a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:48.348921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1159/2363] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:39:48.718849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:48.720354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:48.752689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cf761-3372-4e30-9b48-dcb0228123aa None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:48.758121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:48.759753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:48.760549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:48.761149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] PATH_INFO: `/v3/projects/d154d333ad7947a78489ba93525b1679` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:48.763141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d154d333ad7947a78489ba93525b1679)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:48.766185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:48.858019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd154d333ad7947a78489ba93525b1679', 'request_id': 'req-c9dac417-39f3-4827-aab9-cea4a65f8a0a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:48.864804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd154d333ad7947a78489ba93525b1679', 'request_id': 'req-c9dac417-39f3-4827-aab9-cea4a65f8a0a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:48.876084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd154d333ad7947a78489ba93525b1679', 'request_id': 'req-c9dac417-39f3-4827-aab9-cea4a65f8a0a'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:48.879946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:48.896384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] The token cache is being invalidate because project d154d333ad7947a78489ba93525b1679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:48.897586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:48.908313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:48.910587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1205/2364] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:48 2022] DELETE /identity/v3/projects/d154d333ad7947a78489ba93525b1679 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:49.230882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49768-be57-47e1-b713-306d380f37ee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:49.231406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49768-be57-47e1-b713-306d380f37ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:49.231406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49768-be57-47e1-b713-306d380f37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:49.261963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-53b49768-be57-47e1-b713-306d380f37ee None None] MFA Rules not processed for user `6f774d316a224f30b17e1985a51e8198`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:49.263330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-53b49768-be57-47e1-b713-306d380f37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:49.349627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1160/2365] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:49.360980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:49.362512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:49.470891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac417-39f3-4827-aab9-cea4a65f8a0a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:49.488850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:49.490371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:49.490780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:49.491195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:49.517746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:49.528891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a5b14-f641-4da1-a3b2-a0079de34e0b service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:49.544805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1206/2366] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:52.190826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c64a-d451-4951-8cf0-c9c6e4b666dd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:52.191432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c64a-d451-4951-8cf0-c9c6e4b666dd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:52.191432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c64a-d451-4951-8cf0-c9c6e4b666dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:52.209453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9481c64a-d451-4951-8cf0-c9c6e4b666dd None None] MFA Rules not processed for user `6f774d316a224f30b17e1985a51e8198`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:52.210801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9481c64a-d451-4951-8cf0-c9c6e4b666dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:52.230312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1161/2367] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:59.538393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=28 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:02.204085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=4,keystone:SELECT=83 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:07.928448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:07.929019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:07.929142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:07.947028 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] MFA Rules not processed for user `aa2438a9459d4e9495d4ca19b6bacec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:07.948285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:08.013945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1207/2368] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:40:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:18.006991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=25 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:51.839076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:51.839668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:51.839936 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:51.858202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] MFA Rules not processed for user `6f774d316a224f30b17e1985a51e8198`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:51.859406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:51.871148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1162/2369] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:40:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:53.543034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:53.544946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:53.634427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff07bd1-c4a4-4d79-9f66-a617ad48545c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:53.638756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:53.639861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:53.640098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:53.640524 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:53.642838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:53.741722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:53.747137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecaf1a-c378-49e8-8c70-0fb98bcc2f60 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:53.776175 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1208/2370] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:01.852784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:02.155870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:02.165428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fa6f8-b0e1-4147-b13f-c3cb2d0f8ce2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:02.168960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:02.169600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:02.169880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:02.170155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:02.171919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:02.233386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:02.234810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:02.244230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1163/2371] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:03.759640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=50 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:06.504556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:06.505181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:06.505181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:06.522885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] MFA Rules not processed for user `0f6f9343977343d78848c52d103255f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:06.524266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:06.581471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1209/2372] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:41:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:12.240040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=30 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:16.573884 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=25 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:29.250225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:29.251850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:29.264837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5bd93-31ac-4785-8694-061773dfce43 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:29.268435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:29.269080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:29.269359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:29.269629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:29.271548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:29.345341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:29.347387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:29.359161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1164/2373] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:31.597118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:31.606725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0576901-a85c-4c01-ba82-f5ba6366df28 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:31.613432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:31.614072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:31.614354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:31.614655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:31.616442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:31.631339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:31.632794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0c57-6506-4058-a8a9-1766ffa6c95c service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.646871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1210/2374] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:32.775745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:32.777153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:32.791445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a074-d4d2-4db5-88f1-193ec1359c19 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:32.796994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:32.797636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:32.797928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:32.798204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:32.799948 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:32.861202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:32.862856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:32.871870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1165/2375] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:38.081564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e484-461e-4a23-b6f0-aa4798fe742d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:38.082025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e484-461e-4a23-b6f0-aa4798fe742d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:38.082117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e484-461e-4a23-b6f0-aa4798fe742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:38.100636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9987e484-461e-4a23-b6f0-aa4798fe742d None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:38.101830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9987e484-461e-4a23-b6f0-aa4798fe742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:38.156234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1211/2376] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:38.199863 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:38.201261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:38.277287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5cfbd-e7ad-4b5c-af0a-8d853323f7f0 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:38.282494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:38.283150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:38.283430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:38.283736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:38.292976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:38.294566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:38.303487 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1166/2377] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:41:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:41:48.149545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=31 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:48.299635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:54.308603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54876b80-f1b4-4610-9046-609350aa471b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:54.309108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54876b80-f1b4-4610-9046-609350aa471b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:54.309217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54876b80-f1b4-4610-9046-609350aa471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:54.326765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-54876b80-f1b4-4610-9046-609350aa471b None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:54.327980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-54876b80-f1b4-4610-9046-609350aa471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:54.337514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1212/2378] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:54.351120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:54.361469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8360be-0f9d-49b2-b601-027869fef733 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:54.367173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:54.368004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:54.368299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:54.368579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:54.377097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:54.378669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:54.386734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1167/2379] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:41:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:41:56.304424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54876b80-f1b4-4610-9046-609350aa471b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:56.306020 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-54876b80-f1b4-4610-9046-609350aa471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:56.380958 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54876b80-f1b4-4610-9046-609350aa471b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:56.386240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:56.387112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.387396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.387674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] PATH_INFO: `/v3/users/6f774d316a224f30b17e1985a51e8198` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.388900 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f774d316a224f30b17e1985a51e8198)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:56.390823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:56.442088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f774d316a224f30b17e1985a51e8198'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.454093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f774d316a224f30b17e1985a51e8198'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.454766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f774d316a224f30b17e1985a51e8198'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.455565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f774d316a224f30b17e1985a51e8198'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.462689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c9f20568-6d87-4ae7-8b7b-3e2ef939acc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f774d316a224f30b17e1985a51e8198'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.476985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1213/2380] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:41:56 2022] DELETE /identity/v3/users/6f774d316a224f30b17e1985a51e8198 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:56.657847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:56.671221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedfa2-fd8b-44dd-8247-2fe21da130f0 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:56.731149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:56.732043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.732325 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.732597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] PATH_INFO: `/v3/projects/90731b2dc5a4462184d4baa1de00ebcf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.733756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90731b2dc5a4462184d4baa1de00ebcf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:56.735640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:56.781389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90731b2dc5a4462184d4baa1de00ebcf', 'request_id': 'req-77913adb-64f9-4c32-873a-c0bed3238c67'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.787512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90731b2dc5a4462184d4baa1de00ebcf', 'request_id': 'req-77913adb-64f9-4c32-873a-c0bed3238c67'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.793612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90731b2dc5a4462184d4baa1de00ebcf', 'request_id': 'req-77913adb-64f9-4c32-873a-c0bed3238c67'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.805833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] The token cache is being invalidate because project 90731b2dc5a4462184d4baa1de00ebcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:41:56.806151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.806769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-77913adb-64f9-4c32-873a-c0bed3238c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:56.808331 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1168/2381] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:41:56 2022] DELETE /identity/v3/projects/90731b2dc5a4462184d4baa1de00ebcf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:56.813385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.813705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.813989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.832279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:56.833462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:56.891407 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1214/2382] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:56.899244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.899609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.899986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.919695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:56.920922 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:56.928946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1169/2383] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:56.933062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:56.942664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3ca03-3a1f-4430-82bf-65862c1ae734 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:56.947212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:56.947908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.948179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.948457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.949191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:41:56.949500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:56.951091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:56.960201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:56.964525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1215/2384] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:41:56.968718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:56.978494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfadf26-cba3-48a7-8385-b7fa5dc586e2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:56.983157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:56.983842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:56.984130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:56.984404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:56.985100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:56.986981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.011626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1170/2385] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:41:57.015543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:57.025893 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf9ada-a179-4ad7-a4b7-24a502a79a86 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:57.081797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:57.082478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.082750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.083034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.083730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:57.085673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.106398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1216/2386] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:41:57 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:41:57.110836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:57.121229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4cd4d-bd09-4de0-b7fd-32c1c1e33243 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:57.126111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:57.126766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.127563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.127876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.128603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:57.130194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.140439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1171/2387] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:41:57.145005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:57.154897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a583-4c9c-400c-8caf-215ecbf2d430 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:57.159715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:57.160399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.160685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.160963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.161689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:57.163275 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537cf9c-9bbd-44f0-a5eb-fe9b4a9ca141 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.173097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1217/2388] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:41:57.177310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:57.187083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab818f-b638-4ebd-b0af-264f412e3599 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:57.192121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:57.192984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.193264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.193536 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] PATH_INFO: `/v3/projects/2fe295a5b8074b72b59c0bc5237cbaf4/users/ed51f9aae66e4b64bfac069fee06121d/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.208635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe295a5b8074b72b59c0bc5237cbaf4, user_id=ed51f9aae66e4b64bfac069fee06121d, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:57.211341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.223651 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1172/2389] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:41:57 2022] PUT /identity/v3/projects/2fe295a5b8074b72b59c0bc5237cbaf4/users/ed51f9aae66e4b64bfac069fee06121d/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:57.287152 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.287440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.287747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.313836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] MFA Rules not processed for user `ed51f9aae66e4b64bfac069fee06121d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:57.315472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:57.379132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1218/2390] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:41:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:57.390387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:57.392092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:57.407074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:57.452417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53c191-abb7-4f42-9931-0f5deb7d22e2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:57.456067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:57.456716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:57.457003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:57.457275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:57.465498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:57.466914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf668a4-cd6e-4c28-99af-13412126d67f service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:57.475634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1173/2391] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:07.375707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:07.471633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=20 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:12.805079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:12.868626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4ffc9-eca7-4410-b777-f8f1a1f47182 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:12.874149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:12.875059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:12.875352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:12.875663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] PATH_INFO: `/v3/users/ed51f9aae66e4b64bfac069fee06121d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:12.877021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed51f9aae66e4b64bfac069fee06121d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:12.878973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:12.930361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed51f9aae66e4b64bfac069fee06121d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:12.942255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed51f9aae66e4b64bfac069fee06121d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:12.942933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed51f9aae66e4b64bfac069fee06121d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:12.943550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed51f9aae66e4b64bfac069fee06121d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:12.950379 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed51f9aae66e4b64bfac069fee06121d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:12.963106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1219/2392] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:42:12 2022] DELETE /identity/v3/users/ed51f9aae66e4b64bfac069fee06121d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:12.967825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:12.968318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:12.968585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:12.986382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:12.988160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:13.045691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1174/2393] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:42:13.056062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:13.057460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:13.114011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0dea18-743b-4a14-89f8-c731fea7866f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:13.117666 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:13.118337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:13.118617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:13.118892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:13.128056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:13.129477 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:13.138738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1220/2394] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:42:13.406641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:13.408090 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:13.421856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30f90a-5ec0-4474-9230-e76e70341811 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:13.427209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:13.428120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:13.428408 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:13.428683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] PATH_INFO: `/v3/projects/2fe295a5b8074b72b59c0bc5237cbaf4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:13.430989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe295a5b8074b72b59c0bc5237cbaf4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:13.432843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:13.489663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe295a5b8074b72b59c0bc5237cbaf4', 'request_id': 'req-881bcb06-f301-42ef-b93d-ded8047743da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:13.495146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe295a5b8074b72b59c0bc5237cbaf4', 'request_id': 'req-881bcb06-f301-42ef-b93d-ded8047743da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:13.501470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe295a5b8074b72b59c0bc5237cbaf4', 'request_id': 'req-881bcb06-f301-42ef-b93d-ded8047743da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:13.514202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] The token cache is being invalidate because project 2fe295a5b8074b72b59c0bc5237cbaf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:42:13.514526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:13.515148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:13.516737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1175/2395] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:42:13 2022] DELETE /identity/v3/projects/2fe295a5b8074b72b59c0bc5237cbaf4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:23.134554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:23.511195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:33.221344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:33.223300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:33.238348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856399e-5930-4880-a221-d674639cc200 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:33.317594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:33.318478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:33.318764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:33.319038 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] PATH_INFO: `/v3/users/0f6f9343977343d78848c52d103255f5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:33.321072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6f9343977343d78848c52d103255f5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:33.322983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:33.374912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6f9343977343d78848c52d103255f5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.387571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6f9343977343d78848c52d103255f5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.388330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6f9343977343d78848c52d103255f5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.388946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6f9343977343d78848c52d103255f5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.397377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6f9343977343d78848c52d103255f5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.411623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1221/2396] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:42:33 2022] DELETE /identity/v3/users/0f6f9343977343d78848c52d103255f5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:33.586320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:33.587868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:33.602693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcb06-f301-42ef-b93d-ded8047743da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:33.662628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:33.663488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:33.663783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:33.664070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] PATH_INFO: `/v3/projects/f9124a7a7d7e498b9f7c28671200e203` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:33.665233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9124a7a7d7e498b9f7c28671200e203)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:33.667116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:33.710932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9124a7a7d7e498b9f7c28671200e203', 'request_id': 'req-6680cbc6-eb94-4079-b01e-f95deabfa368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.716317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9124a7a7d7e498b9f7c28671200e203', 'request_id': 'req-6680cbc6-eb94-4079-b01e-f95deabfa368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.722871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9124a7a7d7e498b9f7c28671200e203', 'request_id': 'req-6680cbc6-eb94-4079-b01e-f95deabfa368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.735795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] The token cache is being invalidate because project f9124a7a7d7e498b9f7c28671200e203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:42:33.736635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.737271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:33.739118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1176/2397] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:42:33 2022] DELETE /identity/v3/projects/f9124a7a7d7e498b9f7c28671200e203 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:37.907734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:37.909307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:37.965999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8f2b-4344-4024-b9f0-107d833de151 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:37.999704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:38.021530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:38.022187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:38.022470 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:38.022745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:38.024560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:38.033782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:38.035287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3094f5-584a-4a7f-bd67-38fffc2bc49f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:38.044317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1222/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:43.735131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:48.000007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:48.001441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:48.040161 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=10 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:48.077178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680cbc6-eb94-4079-b01e-f95deabfa368 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:48.082548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:48.083189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:48.083471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:48.083749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:48.085520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:48.097761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:48.099438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:48.109817 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1177/2399] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:50.833484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:50.833994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:50.834100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:50.852989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:50.854186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:50.940251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1223/2400] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:42:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:42:50.963002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:50.964650 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:51.041954 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445ce4f-0be6-4444-838f-307a6b11774b service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:51.046861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:51.047504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:51.047815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:51.048092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:51.056557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:51.058125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad2829-f9bc-4428-85a9-ef908a765e0d service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:51.066020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1178/2401] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:42:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:43:00.928995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=25 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:01.062909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=66 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:07.938971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:07.940382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:07.951452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096020d-d2f6-42a2-8f2d-c3dbe5db395d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:07.955073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:07.955706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:07.956027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:07.956306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:07.957987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:07.967984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:07.969405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245a7b9-ec9d-4690-9302-b1ae8076827b service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:07.977821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1224/2402] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:08.077243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.077845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.077845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.079323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1179/2403] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:43:08.085001 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea09e-2e44-4d7d-8631-ba271c1db186 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.085374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea09e-2e44-4d7d-8631-ba271c1db186 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.085672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea09e-2e44-4d7d-8631-ba271c1db186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.104770 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-42dea09e-2e44-4d7d-8631-ba271c1db186 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:08.106198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-42dea09e-2e44-4d7d-8631-ba271c1db186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.115959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1225/2404] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:08.127603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:08.129360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.196722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b93ca-9bca-49c6-aede-6b3761d3bbcd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:08.200197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:08.200850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.201131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.201414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.212991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:08.214406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739466b5-0f1c-47ea-8b8a-14e6147b48db service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:08.225005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1180/2405] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:08.292567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.292904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.293182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.294547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1226/2406] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:43:08.299157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e403e-519b-443f-bf57-0e9259e9abb5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.299442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e403e-519b-443f-bf57-0e9259e9abb5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.299717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e403e-519b-443f-bf57-0e9259e9abb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.317664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d10e403e-519b-443f-bf57-0e9259e9abb5 None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:08.318846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d10e403e-519b-443f-bf57-0e9259e9abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.326255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1181/2407] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:08.338525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:08.340375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.353419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283af971-91c1-423e-8382-cce4a6be47fe None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:08.356845 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:08.357481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.357747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.358019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.364855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:08.366264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e4cd-09c3-43a0-84a4-bfe8ea3c0373 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:08.374211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1227/2408] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:08.457442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6430b-4be8-4667-9f00-6d7387b3e7a9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.457734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6430b-4be8-4667-9f00-6d7387b3e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.458043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6430b-4be8-4667-9f00-6d7387b3e7a9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.459428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1182/2409] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:43:08.465609 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.465894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.466172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.484333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:08.485518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.495179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1228/2410] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:08.505658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6430b-4be8-4667-9f00-6d7387b3e7a9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:08.517301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6430b-4be8-4667-9f00-6d7387b3e7a9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:08.520881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:08.521509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.521882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.522166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.532140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:08.533553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d6d42-7fa7-4281-acc2-b50ca88814d4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:08.544194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1183/2411] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:08.609702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:08.611094 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.679032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111ddd7-e893-4fa1-ad8c-84027d42e29a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:08.683979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:08.684617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:08.684895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:08.685167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:08.686933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:08.698036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:08.699596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:08.709200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1229/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:11.241655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:11.242108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:11.242258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:11.261315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:11.262437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:11.269466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1184/2413] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:43:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:11.287504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:11.299085 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6587a1-371f-40ec-9e40-66726ec2f7dc service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:11.304073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:11.304710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:11.304986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:11.305259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:11.313241 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:11.314811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c2809-07a1-45b5-b4cf-0d424ffd89db service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:11.323311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1230/2414] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:43:21.254598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=38 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:21.319633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=56 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:33.111124 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:33.112593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:33.128634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d814617-b294-4da7-a0f1-d8667fbfdb71 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:33.134085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:33.134723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:33.135007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:33.135281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:33.137522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:33.197691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:33.199899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:33.209432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1185/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:39.046904 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9682d7-a4ec-434b-9bf4-638085d7c0ee None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:39.047427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9682d7-a4ec-434b-9bf4-638085d7c0ee None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:39.047427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9682d7-a4ec-434b-9bf4-638085d7c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:39.066302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ab9682d7-a4ec-434b-9bf4-638085d7c0ee None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:39.067498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ab9682d7-a4ec-434b-9bf4-638085d7c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:39.075513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1231/2416] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:39.088441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:39.100542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644baae2-1722-4092-8ac4-f476ba6f11ca service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:39.106612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:39.107374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:39.107934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:39.108287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:39.117866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:39.119540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:39.129136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1186/2417] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:43:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:43:49.060715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:49.125540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=34 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:58.474509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:58.475035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:58.475035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:58.492131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] MFA Rules not processed for user `aa2438a9459d4e9495d4ca19b6bacec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:58.493324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:58.500986 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1232/2418] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:43:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:04.514339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:04.527512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016220f-e8a3-4089-9be7-35a68e8b0a9c service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:04.533511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:04.534153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:04.534436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:04.534708 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:04.536473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:04.549662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:04.551235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd7c36-b437-42ac-a836-163d4d4e1cb5 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:04.561715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1187/2419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:08.486919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:14.557299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:20.345691 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:20.358835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44239f2c-eeaf-4345-8a26-97f29daad381 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:20.364346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:20.365058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:20.365348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:20.365648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:20.367423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:20.426871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:20.428437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada6436-ce69-4fa5-810b-daa463016080 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:20.438217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1233/2420] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:30.433385 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=28 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:34.832081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4a4b-c4a1-40c3-a3a2-20f54a62a181 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:34.832698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4a4b-c4a1-40c3-a3a2-20f54a62a181 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:34.832931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c4a4b-c4a1-40c3-a3a2-20f54a62a181 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:34.834725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1188/2421] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:44:34.840693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285daa-1cc3-41c1-9571-2d8692cb57ba None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:34.841136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285daa-1cc3-41c1-9571-2d8692cb57ba None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:34.841392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285daa-1cc3-41c1-9571-2d8692cb57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:34.891797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b5285daa-1cc3-41c1-9571-2d8692cb57ba None None] MFA Rules not processed for user `29a153f06f9e4b459a413f2f936c1f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:34.892977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b5285daa-1cc3-41c1-9571-2d8692cb57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:34.919411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1234/2422] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:44:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:44:34.931928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4a4b-c4a1-40c3-a3a2-20f54a62a181 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:34.943386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c4a4b-c4a1-40c3-a3a2-20f54a62a181 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:34.946938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:34.947614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:34.947935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:34.948211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:34.956704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:34.964767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf197e1-4c04-40e8-92c2-a3afa4c10ffd service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:34.972953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1189/2423] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:35.056597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51442d-8d35-4f0d-a305-db9e89a0271d None None] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:35.056887 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51442d-8d35-4f0d-a305-db9e89a0271d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:35.057164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51442d-8d35-4f0d-a305-db9e89a0271d None None] PATH_INFO: `/` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:35.058533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1235/2424] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:44:35.063301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:35.063584 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:35.063882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:35.093916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:35.095102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:35.139515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1190/2425] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:35.149818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51442d-8d35-4f0d-a305-db9e89a0271d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:35.161370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51442d-8d35-4f0d-a305-db9e89a0271d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:35.164829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:35.165458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:35.165741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:35.166016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:35.175915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:35.177334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:35.187842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1236/2426] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:45.136698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=30 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:45.183242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=30 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:04.353086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:04.364134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c490-d06d-4b69-8e3b-c0ff84c1eb57 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:04.370272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:04.370940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:04.371221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:04.371494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:04.373299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:04.430439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:04.432161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:04.440986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1191/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:06.260390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:06.272049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c9b7-e4df-4950-8ab7-361f88b5cedd service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:06.277440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:06.278067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:06.278348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:06.278621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:06.280395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:06.337670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:06.339251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:06.348146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1237/2428] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:11.081542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:11.093967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ce4dc-78ad-483a-a080-a5eae6eab741 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:11.100130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:11.101208 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:11.101588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:11.101899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:11.103934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:11.121786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:11.123851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb8b83-643b-4f9c-b897-ef67fb8a719a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:11.137278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1192/2429] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:13.277635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:13.279053 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:13.334065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99212bf-1e77-4145-bc8f-f03b8ef4a488 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:13.340021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:13.340687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:13.340956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:13.341357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:13.343032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:13.355278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:13.357579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:13.368243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1238/2430] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:45:16.630355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183231-624e-4a11-b986-1e3f35739995 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:16.630828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183231-624e-4a11-b986-1e3f35739995 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:16.631037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183231-624e-4a11-b986-1e3f35739995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:16.648893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88183231-624e-4a11-b986-1e3f35739995 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:16.650223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88183231-624e-4a11-b986-1e3f35739995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:16.657562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1193/2431] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:45:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:16.707619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:16.721404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c67c-9a72-41e4-9c4f-a395f8f932c5 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:16.726669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:16.727513 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:16.728526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:16.729211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:16.741821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:16.743580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532e196-61d5-4bf6-8254-bd90d4252658 service swift] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:16.753816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1239/2432] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:45:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:45:17.202911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183231-624e-4a11-b986-1e3f35739995 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:17.224594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183231-624e-4a11-b986-1e3f35739995 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:17.229023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:17.229713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:17.229984 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:17.230240 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:17.232131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:17.337616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:17.339428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1021-2747-43c5-8805-fba257c7a177 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:17.361872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1194/2433] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:19.552631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:19.553133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:19.553215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:19.572547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] MFA Rules not processed for user `dac164d6b76a4030ac4ebce2675dc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:19.573763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:19.632133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1240/2434] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:19.738422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:19.738893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:19.738990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:19.756801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] MFA Rules not processed for user `58e0d069005b46268544e890ce08ee2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:19.758011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:19.765064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1195/2435] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:20.245625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:20.255787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c78e-bb27-46d5-bb16-f2c2befcfd75 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:20.259851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:20.260550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:20.260834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:20.261113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:20.264536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:20.291088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:20.293219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:20.305443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1241/2436] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:21.675105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:21.696226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f986bc1-9cbb-4e28-bd7b-ad6e42d11566 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:21.713332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:21.715745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:21.719503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:21.719945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:21.728713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:21.760518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:21.762340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:21.798057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1196/2437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:27.392628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:27.404669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e115-bb46-4139-a8fb-592de01003c7 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:27.409503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:27.410139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:27.410418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:27.410693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:27.412454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:27.433236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:27.434843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:27.445873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1242/2438] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:27.459685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:27.461172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:27.474334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:27.482973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b5b75-2e59-4f77-95c4-647be1510ce9 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:27.487920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:27.488819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:27.489109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:27.489385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] PATH_INFO: `/v3/users/bece6c37b7484f0c8ac24cb74a6e6200` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:27.490634 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] RBAC: Authorizing `identity:delete_user(user_id=bece6c37b7484f0c8ac24cb74a6e6200)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:27.492612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:27.539402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bece6c37b7484f0c8ac24cb74a6e6200'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.540077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bece6c37b7484f0c8ac24cb74a6e6200'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.555080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bece6c37b7484f0c8ac24cb74a6e6200'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.562797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bece6c37b7484f0c8ac24cb74a6e6200'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.563485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bece6c37b7484f0c8ac24cb74a6e6200'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.576273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1197/2439] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:27 2022] DELETE /identity/v3/users/bece6c37b7484f0c8ac24cb74a6e6200 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:27.759718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:27.771854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8378863-a922-4a0d-9a06-0844bd37ca60 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:27.793344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:27.845791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:27.846661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:27.846943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:27.847220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] PATH_INFO: `/v3/projects/bf34c09020ce4f75b07bdfc37c77d82f` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:27.848452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf34c09020ce4f75b07bdfc37c77d82f)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:27.850509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:27.912328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf34c09020ce4f75b07bdfc37c77d82f', 'request_id': 'req-8645a3e9-7a06-4876-a7e8-df66477d822b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.918854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf34c09020ce4f75b07bdfc37c77d82f', 'request_id': 'req-8645a3e9-7a06-4876-a7e8-df66477d822b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.926820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf34c09020ce4f75b07bdfc37c77d82f', 'request_id': 'req-8645a3e9-7a06-4876-a7e8-df66477d822b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.938988 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] The token cache is being invalidate because project bf34c09020ce4f75b07bdfc37c77d82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:27.939386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.940126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:27.941973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1243/2440] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:27 2022] DELETE /identity/v3/projects/bf34c09020ce4f75b07bdfc37c77d82f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:27.945676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:27.947706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:27.960746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e08-8f91-433c-9294-5996dbe411de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.020269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.021153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.021432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.021707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] PATH_INFO: `/v3/users/a25ecb25c98044b6b438098022ee4fcc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.022901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a25ecb25c98044b6b438098022ee4fcc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.025459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.074946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a25ecb25c98044b6b438098022ee4fcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.075603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a25ecb25c98044b6b438098022ee4fcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.088210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a25ecb25c98044b6b438098022ee4fcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.095246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a25ecb25c98044b6b438098022ee4fcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.095921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da3afb49-ed3c-475c-b537-7eac855402ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a25ecb25c98044b6b438098022ee4fcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.098414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:28.109161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1198/2441] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:27 2022] DELETE /identity/v3/users/a25ecb25c98044b6b438098022ee4fcc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:28.283390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.284849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:28.302967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645a3e9-7a06-4876-a7e8-df66477d822b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.372796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.373660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.373946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.374493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] PATH_INFO: `/v3/projects/f2525b377f5c4128ba15bc7bfebca951` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.375678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2525b377f5c4128ba15bc7bfebca951)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.378332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.428960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2525b377f5c4128ba15bc7bfebca951', 'request_id': 'req-0b986eeb-9f5e-4d70-aeca-414362ab8936'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.431784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:28.436966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2525b377f5c4128ba15bc7bfebca951', 'request_id': 'req-0b986eeb-9f5e-4d70-aeca-414362ab8936'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.445768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2525b377f5c4128ba15bc7bfebca951', 'request_id': 'req-0b986eeb-9f5e-4d70-aeca-414362ab8936'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.457901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] The token cache is being invalidate because project f2525b377f5c4128ba15bc7bfebca951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:28.458130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.458906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0b986eeb-9f5e-4d70-aeca-414362ab8936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:28.460520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1244/2442] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:28 2022] DELETE /identity/v3/projects/f2525b377f5c4128ba15bc7bfebca951 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:28.465631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.466110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.466202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.513161 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:28.514377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:28.572291 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1199/2443] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:28.580397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.580718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.581018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.599357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:28.600580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:28.610474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1245/2444] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:28.614691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.624838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461e5d7-baaa-4733-a57a-434bfd162dd4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.629359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.630005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.630284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.630557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.631279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:45:28.631585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.633212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.645261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1200/2445] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:45:28.649214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.658919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4d7cb-4a1c-4024-81ef-1a45738e9dc0 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.663532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.664255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.664529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.664802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.666128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.668034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.695383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1246/2446] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:28 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:45:28.700245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.710801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112f056-aa8f-4159-a8c8-2436f4ad5e39 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.769506 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.770176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.770455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.770731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.771424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.773369 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.792815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1201/2447] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:28 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:45:28.797466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.807772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf3d09-36e1-4a83-8a56-19df7c52dd7b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.812355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.813011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.813298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.813577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.814334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.815985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.826768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1247/2448] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:28.830620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.841140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a94d8-16ba-4e10-9e18-39d558887e9b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.845674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.846318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.846596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.846868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.847583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.849274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fe035-e5fb-4a66-a3ca-0fd0174e19a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.862625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1202/2449] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:28.864402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:28.875148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374c445-4e17-4b5c-8e56-659351b5aba7 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:28.879674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:28.880585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.880866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.881137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] PATH_INFO: `/v3/projects/1bc620581ba8491e90acdb632452d971/users/6e6e201df4224d51b84849e9898a150d/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.896653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc620581ba8491e90acdb632452d971, user_id=6e6e201df4224d51b84849e9898a150d, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:28.899314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:28.916843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1248/2450] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:28 2022] PUT /identity/v3/projects/1bc620581ba8491e90acdb632452d971/users/6e6e201df4224d51b84849e9898a150d/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:28.921657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bd6-3488-485b-b5cb-65c1883a41eb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:28.921949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bd6-3488-485b-b5cb-65c1883a41eb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:28.922227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bd6-3488-485b-b5cb-65c1883a41eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:28.943605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-58942bd6-3488-485b-b5cb-65c1883a41eb None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:28.944809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-58942bd6-3488-485b-b5cb-65c1883a41eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:28.979283 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:29.010299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1203/2451] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:29.021486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:29.023159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:29.081855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182f893-764b-410c-9f4a-819d7d9fdcbc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:29.085512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:29.086156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:29.086437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:29.086711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:29.097265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:29.098699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:29.111000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1249/2452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:34.882705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:34.883157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:34.883263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:34.945899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] MFA Rules not processed for user `6e6e201df4224d51b84849e9898a150d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:34.947615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:35.032898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1204/2453] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:35.037793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.048421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88ee1-b1a9-43e0-8c7c-778da2a1263f service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.053086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.053814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.054163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.054504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.055269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.057813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.085283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1250/2454] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:35 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:35.090066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.101546 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203221a-0250-4366-a1e1-658bff2f53d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.167624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.168533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.168726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.169001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.169676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.171609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.195407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1205/2455] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:35 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:35.199265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.211136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014e1a2-691a-4723-b1b0-ad25b061749a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.217214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.217919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.218220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.218485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.224876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.234814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.272428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1251/2456] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:35.277274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.288004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da6a0e-1193-4ada-9adb-fbc53de88cf5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.305292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.306256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.306751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.307034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] PATH_INFO: `/v3/projects/ff94e39429e84d9381a0b5ad0e15a725/users/ffde35756ff44285bcffa846568af106/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.325228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff94e39429e84d9381a0b5ad0e15a725, user_id=ffde35756ff44285bcffa846568af106, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.328053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.345955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1206/2457] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:35 2022] PUT /identity/v3/projects/ff94e39429e84d9381a0b5ad0e15a725/users/ffde35756ff44285bcffa846568af106/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:35.350076 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.364329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc5cdb-d1ca-4706-864b-7e937e86c76f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.427047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:35.449625 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.450606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.450876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.451456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.452441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.454095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefa994-4465-4832-8b8d-424880329a59 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.473898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1252/2458] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:35.479436 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:35.490694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207d7a-cace-4a3c-86f4-76a17384375d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:35.495933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:35.496842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:35.497228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:35.497514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] PATH_INFO: `/v3/projects/ff94e39429e84d9381a0b5ad0e15a725/users/ffde35756ff44285bcffa846568af106/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:35.500021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff94e39429e84d9381a0b5ad0e15a725, user_id=ffde35756ff44285bcffa846568af106, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:35.503285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:35.521767 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1207/2459] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:35 2022] PUT /identity/v3/projects/ff94e39429e84d9381a0b5ad0e15a725/users/ffde35756ff44285bcffa846568af106/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:45:42.835461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:42.835962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:42.836060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:42.871568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] MFA Rules not processed for user `ffde35756ff44285bcffa846568af106`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:42.872833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:42.970443 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1253/2460] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:42.985199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:42.989399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:43.013698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:43.083625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6df61-4c4f-4458-b0d4-00ce81fedc9b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:43.090262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:43.091270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:43.091270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:43.091458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:43.160199 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:43.162131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:43.177716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1208/2461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:45.820613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:45.822114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:45.890170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46988c98-b873-4e1f-bc87-0e6828050bc1 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:45.894866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:45.895785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:45.896253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:45.896532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:45.898421 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:45.991913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:45.993546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:46.005798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1254/2462] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:50.088840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.090641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:50.112596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10209c22-8905-4133-a3fa-922709cea029 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.117290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.118715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.119000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.119276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] PATH_INFO: `/v3/users/aa2438a9459d4e9495d4ca19b6bacec8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.120501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2438a9459d4e9495d4ca19b6bacec8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.122435 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.170777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2438a9459d4e9495d4ca19b6bacec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.171439 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2438a9459d4e9495d4ca19b6bacec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.185299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2438a9459d4e9495d4ca19b6bacec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.193278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2438a9459d4e9495d4ca19b6bacec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.194027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c69b0934-5da0-488a-b3fc-3a30bf8f2166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2438a9459d4e9495d4ca19b6bacec8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.209013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1209/2463] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:50 2022] DELETE /identity/v3/users/aa2438a9459d4e9495d4ca19b6bacec8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:45:50.389778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.401066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2211-38d7-43ab-ad82-b97f36eb7c93 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.414213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:50.470219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.471098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.471380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.471658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] PATH_INFO: `/v3/projects/1b36bacc49184b3bb8ebcb8ee6eab876` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.472877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b36bacc49184b3bb8ebcb8ee6eab876)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.474773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.522299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b36bacc49184b3bb8ebcb8ee6eab876', 'request_id': 'req-5bfa1648-1639-412c-8362-3ccc600f7335'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.527546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b36bacc49184b3bb8ebcb8ee6eab876', 'request_id': 'req-5bfa1648-1639-412c-8362-3ccc600f7335'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.534972 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b36bacc49184b3bb8ebcb8ee6eab876', 'request_id': 'req-5bfa1648-1639-412c-8362-3ccc600f7335'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.552116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] The token cache is being invalidate because project 1b36bacc49184b3bb8ebcb8ee6eab876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:50.552405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.553093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5bfa1648-1639-412c-8362-3ccc600f7335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:50.554662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1255/2464] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:50 2022] DELETE /identity/v3/projects/1b36bacc49184b3bb8ebcb8ee6eab876 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:45:50.560194 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.560486 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.560760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.578617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:50.580007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:50.608959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:50.645264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1210/2465] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:45:50.652822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.653108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.653383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.673401 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:50.674648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:50.683230 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1256/2466] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:50.687444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.697103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407ebca-02b9-4a2e-b298-0f66604fcbd1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.701621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.702264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.702542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.702814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.703527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:45:50.703862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.705442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.716815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1211/2467] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:45:50.720824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.730729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f34c8-b8e2-4cfb-a101-412148f47095 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.735818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.736498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.736783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.737055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.737765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.739748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.766371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1257/2468] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:50 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:50.770374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.780426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477c8ef-b1c9-44e4-98bf-b488d62fb888 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.840121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.840790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.841071 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.841343 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.842039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.843966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.866723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1212/2469] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:50 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:45:50.867906 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.879935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940368a-a78d-4aef-9b9a-824eb4714e1d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.884592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.885405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.885718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.885994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.886730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.888357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.899848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1258/2470] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:50.903785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.913577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7debe-180a-47fd-8f89-5e9772a0bccb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.918693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.919338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.919619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.919928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.920644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.922223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4e5b-6ddc-448c-b4cf-d803b7de7efd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.932667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1213/2471] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:50.936678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.947614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d560f-ef6e-428a-89e6-7029f18aa16f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.955196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.956268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.956552 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.956829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] PATH_INFO: `/v3/projects/58d15e0816ca401d893705f7239a7a8a/users/c37363bf2f8345f2bc6be5f65aa59afd/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.971735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d15e0816ca401d893705f7239a7a8a, user_id=c37363bf2f8345f2bc6be5f65aa59afd, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.974515 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.988603 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1259/2472] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:50 2022] PUT /identity/v3/projects/58d15e0816ca401d893705f7239a7a8a/users/c37363bf2f8345f2bc6be5f65aa59afd/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:50.999753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:51.000104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:51.000381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:51.028121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] MFA Rules not processed for user `c37363bf2f8345f2bc6be5f65aa59afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:51.029345 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:51.095155 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1214/2473] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:51.106590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:51.108334 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:51.158126 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:51.188328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51400d7-1886-44d2-8da0-205d41327689 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:51.194287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:51.194800 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:51.195083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:51.195355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:51.204080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:51.205685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9060d-f334-4202-928c-bd1bea9a61ae service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:51.217002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1260/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:51.448418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:51.478368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:51.521523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9b3c3-a165-4d45-ba65-ffcb727e2a0d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:51.526373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:51.527287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:51.527563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:51.527868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] PATH_INFO: `/v3/users/c37363bf2f8345f2bc6be5f65aa59afd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:51.529259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37363bf2f8345f2bc6be5f65aa59afd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:51.531032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:51.585138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37363bf2f8345f2bc6be5f65aa59afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:51.585803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37363bf2f8345f2bc6be5f65aa59afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:51.598729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37363bf2f8345f2bc6be5f65aa59afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:51.607333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37363bf2f8345f2bc6be5f65aa59afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:51.608058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37363bf2f8345f2bc6be5f65aa59afd'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:51.622822 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1215/2475] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:51 2022] DELETE /identity/v3/users/c37363bf2f8345f2bc6be5f65aa59afd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:51.629433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:51.629927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:51.630316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:51.650190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:51.651389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:51.714983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1261/2476] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:51.724641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:51.726043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:51.790977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe675d-1803-4576-8b1d-78233a537a0c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:51.794902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:51.795595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:51.795901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:51.796270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:51.807062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:51.808664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b29ab3-aa5b-4e14-9f1e-266ed25e9fb6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:51.819824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1216/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:52.111729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.113254 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:52.128766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2cfaf-89d1-4e8e-a887-ea3c4bdd139d None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.134149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.135077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.135300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.135574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] PATH_INFO: `/v3/projects/58d15e0816ca401d893705f7239a7a8a` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.136791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d15e0816ca401d893705f7239a7a8a)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.139643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.188138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d15e0816ca401d893705f7239a7a8a', 'request_id': 'req-05dba8aa-436c-4447-a219-5ee672dc5b60'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:52.193917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d15e0816ca401d893705f7239a7a8a', 'request_id': 'req-05dba8aa-436c-4447-a219-5ee672dc5b60'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:52.204192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d15e0816ca401d893705f7239a7a8a', 'request_id': 'req-05dba8aa-436c-4447-a219-5ee672dc5b60'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:52.216617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] The token cache is being invalidate because project 58d15e0816ca401d893705f7239a7a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:52.216903 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:52.217536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-05dba8aa-436c-4447-a219-5ee672dc5b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:52.219093 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1262/2478] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:52 2022] DELETE /identity/v3/projects/58d15e0816ca401d893705f7239a7a8a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:45:52.223945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.224425 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.224570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.242235 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:52.243518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:52.287322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:52.306491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1217/2479] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:52.315307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.315597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.315925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.337327 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:52.338541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:52.346482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1263/2480] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:52.350739 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.360602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7da4e4-979b-4189-bcc4-6703efc8f176 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.365210 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.365859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.366146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.366421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.367144 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:45:52.367450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.369644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.383986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1218/2481] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:45:52.388779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.398321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df1e92-f0f7-419a-850f-02bea674a613 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.403000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.403657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.403992 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.404274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.404961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.407317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.435959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1264/2482] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:52 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:52.440216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.450795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628cf75-8b08-40a4-a43e-a0f3ee05b7b7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.507232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.508123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.508462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.508752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.509453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.511385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.532759 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1219/2483] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:52 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:45:52.537138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.548259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c388b-1f37-446c-8bce-e08af37e17e5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.552836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.553469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.553752 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.554026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.554746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.556400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.566663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1265/2484] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:52.570764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.582775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab9347-7ff0-4682-a14e-de2df369b05b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.588007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.588671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.588952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.589226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.589944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.591527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbbaec-3cc8-43b4-9850-b25b0d457e12 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.601337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1220/2485] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:52.606325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.616293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299e64-1e13-4f1b-8718-6792c1dee0a8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.621124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.622018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.622278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.622550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] PATH_INFO: `/v3/projects/5c759e95490e4560bccc03373cb3169d/users/6b98e8ca69c44aa692fd5856555c8e09/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.633902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:52.638747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c759e95490e4560bccc03373cb3169d, user_id=6b98e8ca69c44aa692fd5856555c8e09, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.641925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.660711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1266/2486] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:52 2022] PUT /identity/v3/projects/5c759e95490e4560bccc03373cb3169d/users/6b98e8ca69c44aa692fd5856555c8e09/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:52.666444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbbfc-52e7-4cf8-8d78-7ace50cbe003 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.666735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbbfc-52e7-4cf8-8d78-7ace50cbe003 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.667014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcbbfc-52e7-4cf8-8d78-7ace50cbe003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.684335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dffcbbfc-52e7-4cf8-8d78-7ace50cbe003 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:52.685510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dffcbbfc-52e7-4cf8-8d78-7ace50cbe003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:52.745077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1221/2487] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:52.754940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:52.756681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:52.814382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd0060-456e-425e-9268-13e02215c0af admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:52.818014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:52.818662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:52.818946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:52.819219 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:52.828896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:52.830393 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:52.844080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1267/2488] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:45:53.474085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.474644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.474644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.504215 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] MFA Rules not processed for user `6b98e8ca69c44aa692fd5856555c8e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:53.505427 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:53.560432 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:53.588860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1222/2489] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:53.593338 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.603485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace551-58fd-4f15-ba2c-954f2a28858d service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.608033 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.608693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.608975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.609251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.610132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.612088 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.649197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1268/2490] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:53 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:45:53.650460 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.661579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba608348-be67-4c33-9ce7-1e25b7e79f8c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.732213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.732806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.733092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.733370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.734085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.736015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.757096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1223/2491] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:53 2022] POST /identity/v3/users => generated 309 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:53.761067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.771333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07c34-20a9-421e-b759-327916973f1d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.776061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.776759 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.777042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.777320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.778078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.779729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.792565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1269/2492] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:53.796881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.807658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa8afd-7d2a-41e4-a93f-23027938227f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.812338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.813211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.813497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.813775 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] PATH_INFO: `/v3/projects/c6ec11bb3ca844d7b42b53e176334101/users/f87217c0b0714fa8b3ab272467fee085/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.828998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ec11bb3ca844d7b42b53e176334101, user_id=f87217c0b0714fa8b3ab272467fee085, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.832080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.846766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1224/2493] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:53 2022] PUT /identity/v3/projects/c6ec11bb3ca844d7b42b53e176334101/users/f87217c0b0714fa8b3ab272467fee085/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:53.850891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.862256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1df0c4-e026-4f59-9c70-2468f035fa78 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.927227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.927934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.928220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.928495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.929217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.931318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534b7e-294b-4c50-b021-1eeb74719ea2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.943271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1270/2494] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:53.948066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:53.958646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489ae1-4759-4e9a-b65e-51a1edf29bda admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:53.963247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:53.964157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:53.964444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:53.964723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] PATH_INFO: `/v3/projects/c6ec11bb3ca844d7b42b53e176334101/users/f87217c0b0714fa8b3ab272467fee085/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:53.966674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ec11bb3ca844d7b42b53e176334101, user_id=f87217c0b0714fa8b3ab272467fee085, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:53.969336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:53.981148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1225/2495] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:53 2022] PUT /identity/v3/projects/c6ec11bb3ca844d7b42b53e176334101/users/f87217c0b0714fa8b3ab272467fee085/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:54.635863 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:54.636052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:54.636534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:54.668531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] MFA Rules not processed for user `f87217c0b0714fa8b3ab272467fee085`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:54.669741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:54.720793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:54.749112 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1271/2496] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:55.806312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:55.876900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229e37-098f-4020-884f-5700b6a5440f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:55.882500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:55.884171 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:55.884448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:55.884727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] PATH_INFO: `/v3/users/6b98e8ca69c44aa692fd5856555c8e09` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:55.886451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b98e8ca69c44aa692fd5856555c8e09)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:55.888609 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:55.927044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:55.942717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b98e8ca69c44aa692fd5856555c8e09'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:55.943446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b98e8ca69c44aa692fd5856555c8e09'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:55.957380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b98e8ca69c44aa692fd5856555c8e09'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:55.964929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b98e8ca69c44aa692fd5856555c8e09'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:55.965666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b98e8ca69c44aa692fd5856555c8e09'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:55.978250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1226/2497] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:55 2022] DELETE /identity/v3/users/6b98e8ca69c44aa692fd5856555c8e09 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:45:56.160503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:56.172707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0f561-0fa6-4b5b-a331-b1ac4354bd09 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:56.230654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:56.231519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.231833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.232110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] PATH_INFO: `/v3/projects/5c759e95490e4560bccc03373cb3169d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.233282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c759e95490e4560bccc03373cb3169d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:56.235184 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:56.289449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c759e95490e4560bccc03373cb3169d', 'request_id': 'req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.295755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c759e95490e4560bccc03373cb3169d', 'request_id': 'req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.303174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c759e95490e4560bccc03373cb3169d', 'request_id': 'req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.314765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] The token cache is being invalidate because project 5c759e95490e4560bccc03373cb3169d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:56.315056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.315695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.317267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1272/2498] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:56 2022] DELETE /identity/v3/projects/5c759e95490e4560bccc03373cb3169d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:56.321814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:56.323209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:56.336001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21fd8-33c8-47a6-b1f0-31d7c9f462a4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:56.393880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:56.394663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.394951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.395221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] PATH_INFO: `/v3/users/f87217c0b0714fa8b3ab272467fee085` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.396395 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87217c0b0714fa8b3ab272467fee085)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:56.398306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:56.446569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87217c0b0714fa8b3ab272467fee085'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.447269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87217c0b0714fa8b3ab272467fee085'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.459261 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87217c0b0714fa8b3ab272467fee085'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.466185 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87217c0b0714fa8b3ab272467fee085'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.466875 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12c2c3a5-7b64-4bad-b490-05a997865ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87217c0b0714fa8b3ab272467fee085'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.480962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1227/2499] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:45:56 2022] DELETE /identity/v3/users/f87217c0b0714fa8b3ab272467fee085 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:56.655798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:56.657232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:56.670332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fb0d-2b31-4291-b95e-bb7cb0881eb6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:56.727740 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:56.728634 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.728919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.729193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] PATH_INFO: `/v3/projects/c6ec11bb3ca844d7b42b53e176334101` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.730369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ec11bb3ca844d7b42b53e176334101)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:56.732306 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:56.744140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:56.779233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ec11bb3ca844d7b42b53e176334101', 'request_id': 'req-9109b898-85cc-433f-ae38-c96e89e66f39'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.784639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ec11bb3ca844d7b42b53e176334101', 'request_id': 'req-9109b898-85cc-433f-ae38-c96e89e66f39'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.792228 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ec11bb3ca844d7b42b53e176334101', 'request_id': 'req-9109b898-85cc-433f-ae38-c96e89e66f39'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.803164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] The token cache is being invalidate because project c6ec11bb3ca844d7b42b53e176334101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:45:56.803455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.804108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-9109b898-85cc-433f-ae38-c96e89e66f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:45:56.806250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1273/2500] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:45:56 2022] DELETE /identity/v3/projects/c6ec11bb3ca844d7b42b53e176334101 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:56.811166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.811646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.811753 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.829603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:56.830825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:56.860601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:56.893549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1228/2501] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:56.902576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.902804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.903090 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.921209 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:56.922404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:56.931364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1274/2502] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:56.935606 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:56.945572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d06cf-460a-4ac0-96b0-8ec970ba5238 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:56.950743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:56.951391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.951674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.951978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.952697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:45:56.953008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:56.954678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:56.966355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1229/2503] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:45:56.970613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:56.979973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f9a63-e7ba-4b14-9496-00d6e78883f6 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:56.984642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:56.985285 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:56.985572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:56.985849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:56.986534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:56.988469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.014243 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1275/2504] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:56 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:57.018388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.028542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7e4f-13d1-4844-ba49-6206e8ef5018 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.083662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.084420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.084699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.084974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.085657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.088109 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.107618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1230/2505] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:57 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:57.111775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.122282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916edb-90dc-42df-9226-eacc0ba4fe28 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.127163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.127853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.128148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.128455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.129187 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.130772 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.141034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1276/2506] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:57.144760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.156040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912be334-7a85-49b5-a649-b7b9c33e2968 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.160734 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.161392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.161674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.161952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.162672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.164318 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53f750-5d4b-4d31-a216-458ade35949c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.174784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1231/2507] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:57.179082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.189539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19fd5-0965-433c-a328-8a4e18634da3 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.194178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.195060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.195343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.195617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] PATH_INFO: `/v3/projects/89ac2dc7d38d4795b1c733e67d638a80/users/2c075a9fea8b4d6d91a3d03879da973e/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.211068 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ac2dc7d38d4795b1c733e67d638a80, user_id=2c075a9fea8b4d6d91a3d03879da973e, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.214367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.227082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1277/2508] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:57 2022] PUT /identity/v3/projects/89ac2dc7d38d4795b1c733e67d638a80/users/2c075a9fea8b4d6d91a3d03879da973e/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:57.237371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004009-b66b-4b23-beac-beca755d263f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.237667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004009-b66b-4b23-beac-beca755d263f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.237997 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004009-b66b-4b23-beac-beca755d263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.265727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0d004009-b66b-4b23-beac-beca755d263f None None] MFA Rules not processed for user `2c075a9fea8b4d6d91a3d03879da973e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:57.266938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d004009-b66b-4b23-beac-beca755d263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:57.329717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1232/2509] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:57.334156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.395506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624d4ed-6052-4e75-84ab-3cd3a82464fe admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.400827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.401489 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.401773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.402048 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.402749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.405245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.434067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1278/2510] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:45:57 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:57.435213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d004009-b66b-4b23-beac-beca755d263f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.447422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d004009-b66b-4b23-beac-beca755d263f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.477173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:57.509598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.510303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.510589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.510864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.511563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.513500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.533559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1233/2511] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:45:57 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:45:57.537523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.548286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a0720-6575-4c8e-9b78-5a06b7b31df4 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.552960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.553617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.553901 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.554177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.554922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.556550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.567975 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1279/2512] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:57.572378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.582323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c7eb9-3e55-4161-b2d8-3fdae9d3520c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.587473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.588377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.588660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.588931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] PATH_INFO: `/v3/projects/f009aac98c544c418ed0bef9475efc1d/users/2fbfe02575ad403fba1042c4264c05fa/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.603385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f009aac98c544c418ed0bef9475efc1d, user_id=2fbfe02575ad403fba1042c4264c05fa, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.606061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.620298 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1234/2513] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:57 2022] PUT /identity/v3/projects/f009aac98c544c418ed0bef9475efc1d/users/2fbfe02575ad403fba1042c4264c05fa/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:57.624141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.635109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e84831-795e-4ec2-ab77-e9571f992166 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.643645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:57.695431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.696632 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.696917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.697196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.697920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.699580 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0e3be-6ed8-421a-9a28-4b4801b31ed2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.709391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1280/2514] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:45:57.713528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.723777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7995c-8857-4cf5-8949-d413e0905611 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.728359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.729232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.729509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.729782 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] PATH_INFO: `/v3/projects/f009aac98c544c418ed0bef9475efc1d/users/2fbfe02575ad403fba1042c4264c05fa/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.731714 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f009aac98c544c418ed0bef9475efc1d, user_id=2fbfe02575ad403fba1042c4264c05fa, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.734401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.744879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1235/2515] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:45:57 2022] PUT /identity/v3/projects/f009aac98c544c418ed0bef9475efc1d/users/2fbfe02575ad403fba1042c4264c05fa/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:45:57.755324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.755696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.756044 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.782003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] MFA Rules not processed for user `2fbfe02575ad403fba1042c4264c05fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:45:57.783190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:57.849575 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1281/2516] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:45:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:45:57.860800 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:57.862624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:57.942103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c024a-8285-4134-8a78-5279116435b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:57.946463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:57.947047 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:57.947329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:57.947649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:57.956620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:57.958027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4dfb-30bd-4f19-815f-5025d8a0f8f1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:57.967414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1236/2517] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:45:58.035396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:58.045659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8de47-9146-4889-b95e-9eeaacce5b2f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:58.049296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:58.049934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:58.050220 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:58.050527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:58.106127 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:58.107565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:58.116360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1282/2518] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:07.963380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:08.112408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=86 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:15.083329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a639a61-1582-456b-993d-266bf376368e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:15.083860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a639a61-1582-456b-993d-266bf376368e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:15.083969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a639a61-1582-456b-993d-266bf376368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:15.100518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1a639a61-1582-456b-993d-266bf376368e None None] MFA Rules not processed for user `6e6e201df4224d51b84849e9898a150d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:15.101766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1a639a61-1582-456b-993d-266bf376368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:15.155916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1237/2519] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:25.149296 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:31.155041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:31.156473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:31.211731 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80582b79-df33-4132-bdc6-00e9d3d49dfc service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:31.215911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:31.216547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:31.216826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:31.217096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:31.218938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:31.290673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:31.292149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:31.303134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1283/2520] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:32.474132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a639a61-1582-456b-993d-266bf376368e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:32.485209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a639a61-1582-456b-993d-266bf376368e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:32.488865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:32.489645 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:32.489933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:32.490206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:32.492045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:32.506352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:32.507886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:32.519097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1238/2521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:39.879100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:39.887620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5177be-61cd-4efe-9bc7-fae80161c2a5 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:39.891236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:39.891918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:39.892206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:39.892485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:39.894239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:39.957180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:39.958623 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:39.968938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1284/2522] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:40.256558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:40.258078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:40.268909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb963cb-341a-4c87-bc56-fa5cef5b2249 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:40.272621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:40.273359 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:40.273709 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:40.274065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:40.276418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:40.344146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:40.345684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:40.356528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1239/2523] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:47.255878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:47.259926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:47.273896 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d755b0c-7ed3-489b-81b1-9b3329535367 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:47.279543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:47.280687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:47.281174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:47.281480 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] PATH_INFO: `/v3/users/5b79be1e1cd04b7695373fae08d97039` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:47.283572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b79be1e1cd04b7695373fae08d97039)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:47.285559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:47.322299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:47.337677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b79be1e1cd04b7695373fae08d97039'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.350877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b79be1e1cd04b7695373fae08d97039'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.351582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b79be1e1cd04b7695373fae08d97039'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.352337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b79be1e1cd04b7695373fae08d97039'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.359677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b79be1e1cd04b7695373fae08d97039'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.372912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1285/2524] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:47 2022] DELETE /identity/v3/users/5b79be1e1cd04b7695373fae08d97039 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:47.557280 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:47.569166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4777a05-6492-4e25-be93-0c2bc69afa26 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:47.638577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:47.639217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:47.639531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:47.640505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] PATH_INFO: `/v3/projects/1588191e22ee40979999efc6dfc826bf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:47.641996 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1588191e22ee40979999efc6dfc826bf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:47.644554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:47.700586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1588191e22ee40979999efc6dfc826bf', 'request_id': 'req-86679605-d115-4e75-b1c9-deefd3517bcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.706390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1588191e22ee40979999efc6dfc826bf', 'request_id': 'req-86679605-d115-4e75-b1c9-deefd3517bcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.712935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1588191e22ee40979999efc6dfc826bf', 'request_id': 'req-86679605-d115-4e75-b1c9-deefd3517bcc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.726509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] The token cache is being invalidate because project 1588191e22ee40979999efc6dfc826bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:47.726804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.727480 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.732073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1240/2525] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:47 2022] DELETE /identity/v3/projects/1588191e22ee40979999efc6dfc826bf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:47.733281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:47.734665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:47.748123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f17f-7973-4f0b-a791-9b5c720b6f75 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:47.788567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:47.790006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:47.803747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:47.812370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:47.813860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:47.814155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:47.814433 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] PATH_INFO: `/v3/users/8f1be900789c47daa9d7cd32205177a5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:47.816017 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1be900789c47daa9d7cd32205177a5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:47.818117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:47.859851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679605-d115-4e75-b1c9-deefd3517bcc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:47.864519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:47.865391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:47.865672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:47.865945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] PATH_INFO: `/v3/users/2c075a9fea8b4d6d91a3d03879da973e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:47.867138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c075a9fea8b4d6d91a3d03879da973e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:47.869056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1be900789c47daa9d7cd32205177a5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.869639 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:47.881250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1be900789c47daa9d7cd32205177a5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.881940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1be900789c47daa9d7cd32205177a5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.882547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1be900789c47daa9d7cd32205177a5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.889450 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-947cc4b6-9860-44b7-bb33-18be62f2f5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1be900789c47daa9d7cd32205177a5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.903428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1286/2526] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:47 2022] DELETE /identity/v3/users/8f1be900789c47daa9d7cd32205177a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:47.918893 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c075a9fea8b4d6d91a3d03879da973e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.919582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c075a9fea8b4d6d91a3d03879da973e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.933127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c075a9fea8b4d6d91a3d03879da973e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.940671 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c075a9fea8b4d6d91a3d03879da973e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.941451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c075a9fea8b4d6d91a3d03879da973e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:47.957913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1241/2527] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:47 2022] DELETE /identity/v3/users/2c075a9fea8b4d6d91a3d03879da973e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:46:47.963192 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:47.963558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:47.963954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:47.983851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:47.985173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:48.006381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:48.050124 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1287/2528] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:48.060250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:48.061680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:48.092722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:48.094797 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:48.109245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533c1e1-83cf-4219-afb5-eab2ab8861ce None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:48.114333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:48.115528 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:48.115836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:48.116107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] PATH_INFO: `/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:48.118200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9d395cff7746cdbfef7af5ac421ccd)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:48.120461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:48.132099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a868d9-9862-464d-860f-4a5b3cfc7c59 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:48.136557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:48.137292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:48.137673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:48.137987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:48.151225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:48.152654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:48.162816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1242/2529] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:48.170586 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9d395cff7746cdbfef7af5ac421ccd', 'request_id': 'req-96c5be69-6917-4d32-afea-0d1619d052be'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.176057 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9d395cff7746cdbfef7af5ac421ccd', 'request_id': 'req-96c5be69-6917-4d32-afea-0d1619d052be'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.185937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9d395cff7746cdbfef7af5ac421ccd', 'request_id': 'req-96c5be69-6917-4d32-afea-0d1619d052be'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.199594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] The token cache is being invalidate because project 9a9d395cff7746cdbfef7af5ac421ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:48.199968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.200644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.203805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1288/2530] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:48 2022] DELETE /identity/v3/projects/9a9d395cff7746cdbfef7af5ac421ccd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:48.483207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:48.484886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:48.509001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd046e-efc4-4d7e-8844-3d9beb495678 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:48.540410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:48.567302 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:48.568195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:48.568478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:48.568756 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] PATH_INFO: `/v3/projects/89ac2dc7d38d4795b1c733e67d638a80` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:48.569931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ac2dc7d38d4795b1c733e67d638a80)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:48.572405 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:48.618900 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ac2dc7d38d4795b1c733e67d638a80', 'request_id': 'req-fcf453e6-b583-4662-b214-47cb19e293df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.625415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ac2dc7d38d4795b1c733e67d638a80', 'request_id': 'req-fcf453e6-b583-4662-b214-47cb19e293df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.631482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ac2dc7d38d4795b1c733e67d638a80', 'request_id': 'req-fcf453e6-b583-4662-b214-47cb19e293df'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.644950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] The token cache is being invalidate because project 89ac2dc7d38d4795b1c733e67d638a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:48.645233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.645844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.647389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1243/2531] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:48 2022] DELETE /identity/v3/projects/89ac2dc7d38d4795b1c733e67d638a80 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:48.652013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:48.653411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:48.665997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5be69-6917-4d32-afea-0d1619d052be admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:48.721676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:48.722550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:48.722829 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:48.723096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] PATH_INFO: `/v3/users/2fbfe02575ad403fba1042c4264c05fa` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:48.724298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbfe02575ad403fba1042c4264c05fa)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:48.726202 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:48.787247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbfe02575ad403fba1042c4264c05fa'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.791636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:48.800810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbfe02575ad403fba1042c4264c05fa'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.801596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbfe02575ad403fba1042c4264c05fa'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.802444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbfe02575ad403fba1042c4264c05fa'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.810096 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2bcc38a-e37e-4ab6-92c9-17ac36c758bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbfe02575ad403fba1042c4264c05fa'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:48.823395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1289/2532] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:48 2022] DELETE /identity/v3/users/2fbfe02575ad403fba1042c4264c05fa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:49.089321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.090784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:49.105646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf453e6-b583-4662-b214-47cb19e293df admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.164295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.165164 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.165441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.165715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] PATH_INFO: `/v3/projects/f009aac98c544c418ed0bef9475efc1d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.166953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f009aac98c544c418ed0bef9475efc1d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.168839 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.216572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f009aac98c544c418ed0bef9475efc1d', 'request_id': 'req-780002d8-e8f1-46f0-9bc0-45392241d5ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:49.221953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f009aac98c544c418ed0bef9475efc1d', 'request_id': 'req-780002d8-e8f1-46f0-9bc0-45392241d5ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:49.227870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f009aac98c544c418ed0bef9475efc1d', 'request_id': 'req-780002d8-e8f1-46f0-9bc0-45392241d5ca'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:49.230332 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:49.241518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] The token cache is being invalidate because project f009aac98c544c418ed0bef9475efc1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:49.241802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:49.242417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-780002d8-e8f1-46f0-9bc0-45392241d5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:49.244498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1244/2533] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:49 2022] DELETE /identity/v3/projects/f009aac98c544c418ed0bef9475efc1d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:49.250029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.250360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.250720 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.269140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:49.270314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:49.327951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1290/2534] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:46:49.335920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.336212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.336489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.354643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:49.355860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:49.363594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1245/2535] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:49.368313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.378466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535db33-bfb9-4aee-bc54-9cde04104f5a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.383117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.383787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.384080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.384356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.385086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:46:49.385395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.387128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.398874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1291/2536] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:46:49.402985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.412121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d34e83-eb67-4625-8a52-6c74f8398537 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.417211 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.417785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.418066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.418340 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.419017 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.424824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.458044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1246/2537] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:49.462136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.472133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b4395-d93b-4545-b5b0-d5eb4e1f8236 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.529305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.529974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.530259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.530538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.531255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.533234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.553856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1292/2538] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:49.558527 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.568921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa444b-60a0-4095-9551-a6419455973a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.573560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.574223 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.574507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.574786 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.575514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.577143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.586992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1247/2539] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:49.590687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.600643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a297427-e31c-4283-bd10-7d366b65895f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.605657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.606295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.606574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.606848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.607561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.609170 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f086b1a-cc55-4118-b0d9-68dc7b0ab01a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.619519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1293/2540] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:49.623515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.633664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6312ec-511c-429c-9e1d-be89c3d6b470 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.638392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.639277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.639560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.639873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] PATH_INFO: `/v3/projects/8d4f9b9f010c45cbbf3a85b779b9011f/users/f342dfef14aa461eb8cf90fe7e58526f/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.654507 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4f9b9f010c45cbbf3a85b779b9011f, user_id=f342dfef14aa461eb8cf90fe7e58526f, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.657204 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.670857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1248/2541] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:49 2022] PUT /identity/v3/projects/8d4f9b9f010c45cbbf3a85b779b9011f/users/f342dfef14aa461eb8cf90fe7e58526f/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:49.682084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.682547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.682643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.708852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] MFA Rules not processed for user `f342dfef14aa461eb8cf90fe7e58526f`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:49.710058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:49.734485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:49.775355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1294/2542] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:49.779791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.840600 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8713-a717-43d4-8bed-0475d8eb8b1f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.845228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.845882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.846169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.846446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.847143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.849073 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.873747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1249/2543] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:49.877980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.888542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d8444-74dc-4ee0-96b4-a9043724d862 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.945558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.946672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.947098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.947601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.948576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.950481 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:49.969369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1295/2544] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:49 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:49.973326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:49.983533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018509d0-c897-46e5-b8eb-ea693dce087f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:49.988154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:49.988809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:49.989093 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:49.989371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:49.990088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:49.991663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.003056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1250/2545] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:50.007089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.018105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc98aa-2b14-4e88-a7f7-9fe4413e3b37 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.022746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.023610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.023924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.024199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] PATH_INFO: `/v3/projects/cbaa747fd17d46099fc14facad077495/users/dca3b276c2db49d5a1c56ececc2281c3/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.039084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaa747fd17d46099fc14facad077495, user_id=dca3b276c2db49d5a1c56ececc2281c3, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.041768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.054870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1296/2546] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:50 2022] PUT /identity/v3/projects/cbaa747fd17d46099fc14facad077495/users/dca3b276c2db49d5a1c56ececc2281c3/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:50.059336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.069266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2974-c712-4bc6-9d79-0bdc42b95bc8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.119755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:50.128034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.128897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.129349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.129677 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.130579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.132217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306815f0-09f9-48d5-8bf2-d011ed488566 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.141981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1251/2547] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:50.146080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.156276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203dbc6-9ee7-4b3e-bc5f-7f6a72ad36fc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.160891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.161779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.162047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.162322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] PATH_INFO: `/v3/projects/cbaa747fd17d46099fc14facad077495/users/dca3b276c2db49d5a1c56ececc2281c3/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.164336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaa747fd17d46099fc14facad077495, user_id=dca3b276c2db49d5a1c56ececc2281c3, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.167451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.177542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1297/2548] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:50 2022] PUT /identity/v3/projects/cbaa747fd17d46099fc14facad077495/users/dca3b276c2db49d5a1c56ececc2281c3/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:50.188228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.188522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.188803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.215354 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] MFA Rules not processed for user `dca3b276c2db49d5a1c56ececc2281c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:50.216511 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.291616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1252/2549] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:50.302868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.304630 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.360836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5253394-392a-4f8f-904a-c6bc4d36e6b9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.364479 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.365113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.365397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.365673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.375486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.376943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64649b56-e58a-4eea-979d-4a2786c367e8 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.386259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1298/2550] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:50.414383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.476379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4fa14-ec44-49d0-8343-09711e1f9a33 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.482909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.483894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.484183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.484580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] PATH_INFO: `/v3/users/f342dfef14aa461eb8cf90fe7e58526f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.486398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f342dfef14aa461eb8cf90fe7e58526f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.488370 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.534692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f342dfef14aa461eb8cf90fe7e58526f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:50.535378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f342dfef14aa461eb8cf90fe7e58526f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:50.547722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f342dfef14aa461eb8cf90fe7e58526f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:50.554441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f342dfef14aa461eb8cf90fe7e58526f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:50.555143 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f342dfef14aa461eb8cf90fe7e58526f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:50.558685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:50.570173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1253/2551] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:50 2022] DELETE /identity/v3/users/f342dfef14aa461eb8cf90fe7e58526f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:46:50.574764 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.575101 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.575382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.594300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:50.595496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.608496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:50.654408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1299/2552] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:50.663947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.665923 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.721419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19aba1-6150-4733-8a54-6f2fa61ba57d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.725148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.725831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.726117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.726447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.736319 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.737746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.747115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1254/2553] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:51.018313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.019978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:51.032737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6800db-7e37-4466-98a4-c4b03af3b347 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.037645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.038511 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.038792 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.039069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] PATH_INFO: `/v3/projects/8d4f9b9f010c45cbbf3a85b779b9011f` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.040311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4f9b9f010c45cbbf3a85b779b9011f)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.042225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.089793 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4f9b9f010c45cbbf3a85b779b9011f', 'request_id': 'req-6591b9db-4bc0-412d-b795-854452a08f3d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.095394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4f9b9f010c45cbbf3a85b779b9011f', 'request_id': 'req-6591b9db-4bc0-412d-b795-854452a08f3d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.103813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4f9b9f010c45cbbf3a85b779b9011f', 'request_id': 'req-6591b9db-4bc0-412d-b795-854452a08f3d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.115021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] The token cache is being invalidate because project 8d4f9b9f010c45cbbf3a85b779b9011f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:51.115307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.115965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.117501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1300/2554] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:51 2022] DELETE /identity/v3/projects/8d4f9b9f010c45cbbf3a85b779b9011f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:51.121532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.131661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11b9e-8def-48c4-9b38-2dd9654a40e4 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.189886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.190747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.191029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.191365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] PATH_INFO: `/v3/users/dca3b276c2db49d5a1c56ececc2281c3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.192605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca3b276c2db49d5a1c56ececc2281c3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.194510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.238466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca3b276c2db49d5a1c56ececc2281c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.239104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca3b276c2db49d5a1c56ececc2281c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.251681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca3b276c2db49d5a1c56ececc2281c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.259126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca3b276c2db49d5a1c56ececc2281c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.259778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bf0dbe22-0913-4085-b656-249543186fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca3b276c2db49d5a1c56ececc2281c3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.272118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1255/2555] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:51 2022] DELETE /identity/v3/users/dca3b276c2db49d5a1c56ececc2281c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:51.557491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.558966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:51.572716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591b9db-4bc0-412d-b795-854452a08f3d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.632183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.633097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.633409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.633686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] PATH_INFO: `/v3/projects/cbaa747fd17d46099fc14facad077495` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.634866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaa747fd17d46099fc14facad077495)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.636803 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.681713 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:51.685032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaa747fd17d46099fc14facad077495', 'request_id': 'req-239301b0-3a22-4f65-90ab-f05b6ac89a26'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.692363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaa747fd17d46099fc14facad077495', 'request_id': 'req-239301b0-3a22-4f65-90ab-f05b6ac89a26'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.701504 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaa747fd17d46099fc14facad077495', 'request_id': 'req-239301b0-3a22-4f65-90ab-f05b6ac89a26'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.714130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] The token cache is being invalidate because project cbaa747fd17d46099fc14facad077495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:51.714446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.715948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-239301b0-3a22-4f65-90ab-f05b6ac89a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:51.717707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1301/2556] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:51 2022] DELETE /identity/v3/projects/cbaa747fd17d46099fc14facad077495 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:51.722384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.722889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.723082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.740901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:51.742137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:51.755434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:51.814438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1256/2557] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:46:51.822217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.822507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.822778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.841196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:51.842437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:51.850376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1302/2558] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:51.855156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.865410 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f688dc-82a0-4d2f-8a9f-b46f254af7c7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.874246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.875182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.875464 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.875738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.876484 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:46:51.876788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.878383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.890267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1257/2559] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:46:51.894868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.904743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6457ce-8904-4141-976c-6ce4b5ffccec None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.909465 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.910118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.910397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.910756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.911620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.913840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.942416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1303/2560] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:51 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:51.947061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.958268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe61e-33d8-4f92-8a62-d67f06e2f576 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.014764 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.015530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.015924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.016222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.016907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.018810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.038513 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1258/2561] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:51 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:52.043404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.053965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52debf11-661a-4c21-826d-2b39b417acbc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.058763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.059431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.059852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.060083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.060843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.062501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.073318 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1304/2562] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:52.077262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.087813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c294-b86e-4802-bb74-52d72d43b97b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.092488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.093184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.093472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.093751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.094598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.096232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c0571-4688-44a9-9a8a-83a26e2524d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.107253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1259/2563] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:52.111622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.121661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4a1ad-5e99-402e-b2ef-0aecd1125859 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.126522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.127388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.127669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.127984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] PATH_INFO: `/v3/projects/206dc672cf2242a89018db1c597dd2f2/users/3849e5cc63ec479a9b511a34cc7899c6/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.142485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=206dc672cf2242a89018db1c597dd2f2, user_id=3849e5cc63ec479a9b511a34cc7899c6, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.145207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.158234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1305/2564] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:52 2022] PUT /identity/v3/projects/206dc672cf2242a89018db1c597dd2f2/users/3849e5cc63ec479a9b511a34cc7899c6/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:52.221083 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.221406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.221688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.250898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] MFA Rules not processed for user `3849e5cc63ec479a9b511a34cc7899c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:52.252123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:52.316255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1260/2565] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:52.320660 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.381163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a62ff-7530-44be-909b-6a5360a5ded0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.386370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.387026 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.387313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.387596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.388313 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.390208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.415598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1306/2566] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:52 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:52.419554 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.430691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ee05a-9dbf-4f31-9671-e282870507cf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.440039 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:52.488013 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.488690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.488977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.489254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.489955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.491889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.519191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1261/2567] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:52 2022] POST /identity/v3/users => generated 306 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:52.523214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.534431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864a308-435b-44ba-b9da-69d2e5ce9210 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.539040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.539741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.540051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.540330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.541063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.542646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.553567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1307/2568] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:52.557385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.567294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b44a9-c0c2-4d00-895a-e457f65d2f77 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.572264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.573132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.573417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.573695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] PATH_INFO: `/v3/projects/03e172da460a405eb518c9ed982feef2/users/a6cba39b3c294d4e88402d7ada15f506/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.588745 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e172da460a405eb518c9ed982feef2, user_id=a6cba39b3c294d4e88402d7ada15f506, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.591433 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.603887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1262/2569] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:52 2022] PUT /identity/v3/projects/03e172da460a405eb518c9ed982feef2/users/a6cba39b3c294d4e88402d7ada15f506/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:52.607692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.617546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf9c8-6d95-4609-ab0e-90241ba387d6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.659409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:52.676179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.676849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.677137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.677416 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.678143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.679724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a3445-4055-4d46-9e66-2ee040add650 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.690113 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1308/2570] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:52.694333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.705065 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdd7c-fbfd-469b-91e5-1e36ffd9457b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.709643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.710510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.710792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.711098 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] PATH_INFO: `/v3/projects/03e172da460a405eb518c9ed982feef2/users/a6cba39b3c294d4e88402d7ada15f506/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.713121 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e172da460a405eb518c9ed982feef2, user_id=a6cba39b3c294d4e88402d7ada15f506, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.715799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.725818 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1263/2571] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:52 2022] PUT /identity/v3/projects/03e172da460a405eb518c9ed982feef2/users/a6cba39b3c294d4e88402d7ada15f506/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:52.735744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.736070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.736356 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.763837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] MFA Rules not processed for user `a6cba39b3c294d4e88402d7ada15f506`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:52.765067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:52.832850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1309/2572] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:46:52.843346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:52.845042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:52.902707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e25418-8dfd-4b40-a79c-e94383cc8b21 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:52.906375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:52.907015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:52.907301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:52.907576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:52.916477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:52.917907 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:52.927230 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1264/2573] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:53.001658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:53.012171 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2a12a-3413-4218-b561-8028bc41c513 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:53.015695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:53.016375 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:53.016659 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:53.016935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:53.073649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:53.075249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2786807-829c-49f8-ab21-3056d18695b3 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:53.084047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1310/2574] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:57.105665 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:57.167570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eac77-84b6-4351-a498-75e094959ff7 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:57.172100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:57.172963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:57.173245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:57.173520 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] PATH_INFO: `/v3/users/3849e5cc63ec479a9b511a34cc7899c6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:57.174692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3849e5cc63ec479a9b511a34cc7899c6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:57.176664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:57.187224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:57.226141 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3849e5cc63ec479a9b511a34cc7899c6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.226796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3849e5cc63ec479a9b511a34cc7899c6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.239344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3849e5cc63ec479a9b511a34cc7899c6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.246162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3849e5cc63ec479a9b511a34cc7899c6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.246825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3849e5cc63ec479a9b511a34cc7899c6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.259617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1265/2575] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:57 2022] DELETE /identity/v3/users/3849e5cc63ec479a9b511a34cc7899c6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:57.264163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:57.264644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:57.264750 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:57.282274 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:57.283454 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:57.340790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1311/2576] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:57.350368 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:57.351813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:57.409088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a29c-dc6e-47b0-ab57-2df2c7fe7a63 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:57.412696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:57.413337 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:57.413624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:57.413901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:57.423693 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:57.425166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:57.435572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1266/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:57.705142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:57.706747 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:57.716642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:57.721308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320636-89fd-47fd-aa46-c637ab784669 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:57.726061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:57.727082 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:57.727536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:57.728507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] PATH_INFO: `/v3/projects/206dc672cf2242a89018db1c597dd2f2` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:57.729849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=206dc672cf2242a89018db1c597dd2f2)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:57.731879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:57.779941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '206dc672cf2242a89018db1c597dd2f2', 'request_id': 'req-0841f6c4-0001-49db-95cb-a46435b4e470'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.785461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '206dc672cf2242a89018db1c597dd2f2', 'request_id': 'req-0841f6c4-0001-49db-95cb-a46435b4e470'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.791832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '206dc672cf2242a89018db1c597dd2f2', 'request_id': 'req-0841f6c4-0001-49db-95cb-a46435b4e470'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.802577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] The token cache is being invalidate because project 206dc672cf2242a89018db1c597dd2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:57.802865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.803560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.805129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1312/2578] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:57 2022] DELETE /identity/v3/projects/206dc672cf2242a89018db1c597dd2f2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:57.809173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:57.818827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bedf-b6a2-484b-92c6-2aaab7979a4d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:57.876245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:57.877115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:57.877396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:57.877670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] PATH_INFO: `/v3/users/a6cba39b3c294d4e88402d7ada15f506` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:57.878853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cba39b3c294d4e88402d7ada15f506)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:57.880793 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:57.914588 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:57.929170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cba39b3c294d4e88402d7ada15f506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.929828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cba39b3c294d4e88402d7ada15f506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.942158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cba39b3c294d4e88402d7ada15f506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.949742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cba39b3c294d4e88402d7ada15f506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.950394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9dcfa632-39f9-426e-8651-ff42fcc220a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cba39b3c294d4e88402d7ada15f506'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:57.963133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1267/2579] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:46:57 2022] DELETE /identity/v3/users/a6cba39b3c294d4e88402d7ada15f506 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:46:58.242636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.244107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:58.258686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f6c4-0001-49db-95cb-a46435b4e470 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.319307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.320116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.320399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.320675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] PATH_INFO: `/v3/projects/03e172da460a405eb518c9ed982feef2` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.321864 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e172da460a405eb518c9ed982feef2)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.323823 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.371467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e172da460a405eb518c9ed982feef2', 'request_id': 'req-929c348f-0fb5-4905-abb5-c7ad20119338'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:58.377516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e172da460a405eb518c9ed982feef2', 'request_id': 'req-929c348f-0fb5-4905-abb5-c7ad20119338'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:58.384707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e172da460a405eb518c9ed982feef2', 'request_id': 'req-929c348f-0fb5-4905-abb5-c7ad20119338'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:58.395902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] The token cache is being invalidate because project 03e172da460a405eb518c9ed982feef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:46:58.396185 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:58.396804 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-929c348f-0fb5-4905-abb5-c7ad20119338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:46:58.398331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1313/2580] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:46:58 2022] DELETE /identity/v3/projects/03e172da460a405eb518c9ed982feef2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:58.403267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.403741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.403882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.421951 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:58.423156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:58.483414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1268/2581] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:58.491236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.491529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.491831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.518861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:58.520116 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:58.528526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1314/2582] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:58.532698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.542897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2ca83-a653-415b-ab67-f82edca4285f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.547510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.548201 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.548479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.548749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.549466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:46:58.549778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.552250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.570179 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1269/2583] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:46:58.574366 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.583594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d639fba-b12e-448f-b537-b20cd483175c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.588270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.588915 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.589250 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.589535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.590216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.592133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.600196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:58.619103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1315/2584] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:58.623717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.634099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171fef5-b0fb-4053-9a6a-1026e467cdc5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.690824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.691493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.691801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.692085 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.692783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.694698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.713262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1270/2585] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:58.718021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.728291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c35316-3cc5-4ede-a9f8-9980efab4e6e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.733500 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.734153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.734435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.734710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.735432 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.737052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.747139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1316/2586] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:58.751429 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.761596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d9564-4eb7-4dad-904a-7eb12e579a52 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.766184 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.766865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.767170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.767445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.768236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.769804 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e7491-48fc-429b-a358-993c33fa11cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.780336 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1271/2587] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:58.784341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:58.794937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cf142-277d-4f9c-a0b9-8d7a8855b3a9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:58.799536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:58.800439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.800724 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.800999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] PATH_INFO: `/v3/projects/22eedf78a873489b8ac057091c452f5a/users/83834db18cdb415c9ba663ecbb9d3fd5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.815602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22eedf78a873489b8ac057091c452f5a, user_id=83834db18cdb415c9ba663ecbb9d3fd5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:58.818340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:58.831354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1317/2588] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:58 2022] PUT /identity/v3/projects/22eedf78a873489b8ac057091c452f5a/users/83834db18cdb415c9ba663ecbb9d3fd5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:58.843269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:58.843557 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:58.843861 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:58.870078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] MFA Rules not processed for user `83834db18cdb415c9ba663ecbb9d3fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:58.871259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:58.875378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:58.938470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1272/2589] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:46:58.943233 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.004359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7a07-b047-40b8-9869-35fb0f175a8d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.008897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.009547 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.009827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.010100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.010773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.013190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.037870 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1318/2590] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:58 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:59.041949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.052424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea7f75-c3a5-4b74-af47-820f14139d98 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.106623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.107206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.107485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.107754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.108515 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.110406 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.128977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1273/2591] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:59 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:59.132977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.143801 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d7f52-8214-46d1-a84e-ab4df5f8ebef admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.148364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.149046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.149329 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.149610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.150360 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.152545 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.162876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1319/2592] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:59.166637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.176419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84635b-de26-4ec1-972e-a1714e55cd3f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.180920 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.181561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.181840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.182111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.182841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.184458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d4814-0245-4413-8954-2ec135a37a71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.194064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1274/2593] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:59.198004 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.208267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9b741-9903-43a8-ade5-041c9ee6ec8b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.212832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.213688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.214000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.214812 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] PATH_INFO: `/v3/projects/d4f4dff4a2054207927f2459fcfec854/users/91ab7f5b1c154c28910981a8b984e172/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.229853 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f4dff4a2054207927f2459fcfec854, user_id=91ab7f5b1c154c28910981a8b984e172, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.232527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.244965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1320/2594] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:59 2022] PUT /identity/v3/projects/d4f4dff4a2054207927f2459fcfec854/users/91ab7f5b1c154c28910981a8b984e172/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:46:59.255730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.256050 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.256356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.282944 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] MFA Rules not processed for user `91ab7f5b1c154c28910981a8b984e172`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:59.284192 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:59.346966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1275/2595] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:59.351419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.413995 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae2b84-05bf-45e2-9f2b-b9fe2678f147 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.419196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.419882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.420240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.420434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.421157 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.423049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.431947 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:59.449810 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1321/2596] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:46:59 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:46:59.453724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.464905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30d21-067f-4a5f-b936-92781ff85941 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.473562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:59.520965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.521623 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.521911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.522186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.522882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.524830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.544202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1276/2597] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:46:59 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:46:59.548134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.558352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b0e6-4cfa-4399-8e76-b0062c89c884 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.562857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.563529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.563868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.564146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.564865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.566466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.576762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1322/2598] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:59.580659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.591200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19268b6e-4a4c-4c7b-9c36-cef0b57140d7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.595781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.596659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.596942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.597221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] PATH_INFO: `/v3/projects/5b810e64aa374772946036557ea9e54a/users/e25ec0e567b64468bc2152bc347fbf10/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.612305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b810e64aa374772946036557ea9e54a, user_id=e25ec0e567b64468bc2152bc347fbf10, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.614959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.628250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1277/2599] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:59 2022] PUT /identity/v3/projects/5b810e64aa374772946036557ea9e54a/users/e25ec0e567b64468bc2152bc347fbf10/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:59.632091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.642035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa36fd5-56aa-4a09-88b4-57e21c9da3cc admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.697897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.698493 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.698779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.699060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.699809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.701397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382922c-168b-4048-9f57-1672734262c4 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.711154 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1323/2600] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:46:59.715784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.726016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe0c4c-1e88-438d-85c2-b086075b38e8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.731193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.732103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.732384 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.732657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] PATH_INFO: `/v3/projects/5b810e64aa374772946036557ea9e54a/users/e25ec0e567b64468bc2152bc347fbf10/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.734612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b810e64aa374772946036557ea9e54a, user_id=e25ec0e567b64468bc2152bc347fbf10, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.737241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.747271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1278/2601] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:46:59 2022] PUT /identity/v3/projects/5b810e64aa374772946036557ea9e54a/users/e25ec0e567b64468bc2152bc347fbf10/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:46:59.757631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.757927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.758212 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.785155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] MFA Rules not processed for user `e25ec0e567b64468bc2152bc347fbf10`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:59.786371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:59.857859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1324/2602] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:59.869008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:59.870743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:59.928042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f20a0-3c82-46d8-89e4-3dd6275f0c97 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:59.931678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:59.932349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:59.932635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:59.932966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:59.989575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:59.990917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:59.999742 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1279/2603] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:01.482430 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:01.483927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:01.496234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f149d84-7c3b-43a3-8cc1-f67db61c2a56 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:01.499702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:01.500690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:01.500973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:01.501246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:01.556923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:01.558966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:01.562485 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:01.570055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1325/2604] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:02.093598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:02.102213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67293add-924d-4a9f-bbf0-86ff1f8f9122 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:02.105846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:02.106479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:02.106761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:02.107038 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:02.116003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:02.117412 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7e733-863b-41a4-9e44-5e004bda9c1b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:02.127019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1280/2605] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:05.961924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:06.023976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188bc48-28c1-4ea7-9d09-b68f871338ac service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:06.028665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:06.029525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:06.030302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:06.030578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] PATH_INFO: `/v3/users/83834db18cdb415c9ba663ecbb9d3fd5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:06.031754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83834db18cdb415c9ba663ecbb9d3fd5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:06.033742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:06.078429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83834db18cdb415c9ba663ecbb9d3fd5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.091816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83834db18cdb415c9ba663ecbb9d3fd5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.092991 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83834db18cdb415c9ba663ecbb9d3fd5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.093592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83834db18cdb415c9ba663ecbb9d3fd5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.100392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83834db18cdb415c9ba663ecbb9d3fd5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.113766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1326/2606] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:47:05 2022] DELETE /identity/v3/users/83834db18cdb415c9ba663ecbb9d3fd5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:06.118466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:06.118960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:06.119277 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:06.132153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:06.138585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:06.139899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:06.199539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1281/2607] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:06.209225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:06.210639 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:06.268396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee306-24ca-4178-9f14-dc04d20298c0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:06.272042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:06.272695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:06.272977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:06.273251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:06.282682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:06.284151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:06.293466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1327/2608] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:06.564933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:06.566386 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:06.579043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c0259-65c5-476c-a209-22d9f2632db2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:06.583940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:06.584837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:06.585120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:06.585394 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] PATH_INFO: `/v3/projects/22eedf78a873489b8ac057091c452f5a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:06.586579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22eedf78a873489b8ac057091c452f5a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:06.588587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:06.636396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22eedf78a873489b8ac057091c452f5a', 'request_id': 'req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.642447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22eedf78a873489b8ac057091c452f5a', 'request_id': 'req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.648502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22eedf78a873489b8ac057091c452f5a', 'request_id': 'req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.661011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] The token cache is being invalidate because project 22eedf78a873489b8ac057091c452f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:47:06.661297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.661909 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.663442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1282/2609] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:47:06 2022] DELETE /identity/v3/projects/22eedf78a873489b8ac057091c452f5a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:06.667527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:06.676967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c24da-6166-4fd9-9785-bc3da3b77963 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:06.692767 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:06.734572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:06.735440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:06.735718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:06.736025 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] PATH_INFO: `/v3/users/91ab7f5b1c154c28910981a8b984e172` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:06.737452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ab7f5b1c154c28910981a8b984e172)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:06.739410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:06.783990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ab7f5b1c154c28910981a8b984e172'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.795546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ab7f5b1c154c28910981a8b984e172'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.796267 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ab7f5b1c154c28910981a8b984e172'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.796930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ab7f5b1c154c28910981a8b984e172'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.803891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ab7f5b1c154c28910981a8b984e172'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:06.816902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1328/2610] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:47:06 2022] DELETE /identity/v3/users/91ab7f5b1c154c28910981a8b984e172 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:07.087078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:07.088518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:07.104532 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a4c07-bf8d-4aa1-ac4d-a2918a4e17a5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:07.162668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:07.163525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.163832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.164106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] PATH_INFO: `/v3/projects/d4f4dff4a2054207927f2459fcfec854` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.165284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f4dff4a2054207927f2459fcfec854)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:07.167174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:07.201193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:07.215125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f4dff4a2054207927f2459fcfec854', 'request_id': 'req-be9ae6ae-4007-47e3-8d72-c6df26d6edee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.221797 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f4dff4a2054207927f2459fcfec854', 'request_id': 'req-be9ae6ae-4007-47e3-8d72-c6df26d6edee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.227994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f4dff4a2054207927f2459fcfec854', 'request_id': 'req-be9ae6ae-4007-47e3-8d72-c6df26d6edee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.239840 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] The token cache is being invalidate because project d4f4dff4a2054207927f2459fcfec854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:47:07.240125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.240747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.242263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1283/2611] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:47:07 2022] DELETE /identity/v3/projects/d4f4dff4a2054207927f2459fcfec854 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:07.246232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:07.247684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:07.261476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c10b2-7239-4874-baa4-cb371fa65353 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:07.317151 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:07.318031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.318341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.318617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] PATH_INFO: `/v3/users/e25ec0e567b64468bc2152bc347fbf10` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.319902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25ec0e567b64468bc2152bc347fbf10)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:07.321912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:07.370738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:07.376816 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25ec0e567b64468bc2152bc347fbf10'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.390354 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25ec0e567b64468bc2152bc347fbf10'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.391014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25ec0e567b64468bc2152bc347fbf10'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.391610 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25ec0e567b64468bc2152bc347fbf10'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.398291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca162cdf-e527-425d-9249-3e4dbab151a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25ec0e567b64468bc2152bc347fbf10'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.410865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1329/2612] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:47:07 2022] DELETE /identity/v3/users/e25ec0e567b64468bc2152bc347fbf10 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:07.690705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:07.692200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:07.704717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ae6ae-4007-47e3-8d72-c6df26d6edee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:07.761510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:07.762365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.762644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.762915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] PATH_INFO: `/v3/projects/5b810e64aa374772946036557ea9e54a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.764148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b810e64aa374772946036557ea9e54a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:07.766043 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:07.812752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b810e64aa374772946036557ea9e54a', 'request_id': 'req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.818361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b810e64aa374772946036557ea9e54a', 'request_id': 'req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.824514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b810e64aa374772946036557ea9e54a', 'request_id': 'req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.836393 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] The token cache is being invalidate because project 5b810e64aa374772946036557ea9e54a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:47:07.836679 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.837546 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8c03141b-b8f0-4447-aaa4-c815c6af8ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:07.839088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1284/2613] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:47:07 2022] DELETE /identity/v3/projects/5b810e64aa374772946036557ea9e54a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:07.844535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.844852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.845135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.863092 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:07.864322 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:07.923271 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1330/2614] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:07.931538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.931860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.932170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.950437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:07.952149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:07.959979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1285/2615] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:07.964135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:07.974381 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea295d2-107d-4627-8eb6-5905c7696cc2 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:07.978856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:07.979534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:07.979847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:07.980123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:07.980841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:47:07.981148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:07.982744 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:07.994208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1331/2616] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:47:07.998498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.007972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333caae-ab70-4567-bb6a-e9dee19d12fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.012570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.013214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.013497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.013774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.014986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.016901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.041751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1286/2617] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:47:07 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:47:08.045920 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.055670 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c72674-f588-47ff-82bc-cf36c2aaa2c1 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.110811 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.111472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.111755 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.112060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.113300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.114637 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.134411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1332/2618] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:47:08 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:47:08.138418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.149426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fd4a-5565-47b6-9281-91064545f454 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.154070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.154792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.155669 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.155982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.156706 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.158293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.168783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1287/2619] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:47:08.172841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.182913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7b49-b2b4-4799-bdde-ecb3e8e40cc6 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.188060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.189055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.189055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.189237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.189971 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.191567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307df06-23b4-4501-8c59-faba464c10b1 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.202382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1333/2620] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:47:08.206641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.216631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c1d2-f17a-4b36-96bc-b8dc1c798910 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.221842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.222712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.222994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.223269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] PATH_INFO: `/v3/projects/211606cc420342be8deb3eb71c296ed9/users/ae79a5e2ac034d228252b8c0c20bf2a1/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.231154 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:08.239209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=211606cc420342be8deb3eb71c296ed9, user_id=ae79a5e2ac034d228252b8c0c20bf2a1, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.242146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.255246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1288/2621] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:47:08 2022] PUT /identity/v3/projects/211606cc420342be8deb3eb71c296ed9/users/ae79a5e2ac034d228252b8c0c20bf2a1/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:08.266078 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.266536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.266645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.293336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] MFA Rules not processed for user `ae79a5e2ac034d228252b8c0c20bf2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:08.294535 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:08.306103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:08.363213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1334/2622] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:08.375966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:08.377648 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:08.434163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d1632-3a1f-4520-8386-0a43b1bca501 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:08.438349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:08.439001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:08.439285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:08.439563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:08.448104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:08.449528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:08.459049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1289/2623] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:18.360133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=21 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:18.454926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:43.885907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:43.887445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:43.944316 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330de75b-3750-44fa-a4ab-9786f63154c8 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:43.948347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:43.948997 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:43.949279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:43.949554 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:43.951397 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:43.962186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:43.963699 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e868df-dbbe-4b49-ab2b-9e0d02e3573f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:43.971959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1335/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:53.968183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=26 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:56.269498 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:56.330949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77479-f094-4c68-b875-25f94b5c71fb service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:56.335404 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:56.336289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:56.336565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:56.336836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] PATH_INFO: `/v3/users/ae79a5e2ac034d228252b8c0c20bf2a1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:56.338018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae79a5e2ac034d228252b8c0c20bf2a1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:56.339960 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:56.385917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae79a5e2ac034d228252b8c0c20bf2a1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:56.386595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae79a5e2ac034d228252b8c0c20bf2a1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:56.398324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae79a5e2ac034d228252b8c0c20bf2a1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:56.405133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae79a5e2ac034d228252b8c0c20bf2a1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:56.405761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae79a5e2ac034d228252b8c0c20bf2a1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:56.419585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1290/2625] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:47:56 2022] DELETE /identity/v3/users/ae79a5e2ac034d228252b8c0c20bf2a1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:56.424119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:56.424619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:56.424737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:56.442614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:56.443916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:56.454857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:56.456313 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:56.505095 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1336/2626] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:56.514710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:56.516040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:56.535751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088a263-a4e6-44d2-a16b-7209450c8d0e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:56.542089 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:56.542847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:56.543137 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:56.543415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:56.545245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:56.560294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:56.577594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513289a-df03-44f2-91eb-2a718b394d93 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:56.582280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:56.583135 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:56.583410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:56.584276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:56.594229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:56.596538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:56.607029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1337/2627] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:56.641514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:56.643099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcf987-549e-46d9-af4b-6322b3e9a13a service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:56.661978 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1291/2628] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:47:56.904087 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:56.906103 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:56.919745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4cdd-a5fd-4cbb-8f83-88d7e680da72 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:56.925808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:56.926605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:56.926892 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:56.927193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] PATH_INFO: `/v3/projects/211606cc420342be8deb3eb71c296ed9` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:56.928400 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=211606cc420342be8deb3eb71c296ed9)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:56.930462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:56.990678 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211606cc420342be8deb3eb71c296ed9', 'request_id': 'req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:57.000640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211606cc420342be8deb3eb71c296ed9', 'request_id': 'req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:57.008291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211606cc420342be8deb3eb71c296ed9', 'request_id': 'req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:57.019701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] The token cache is being invalidate because project 211606cc420342be8deb3eb71c296ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:47:57.020034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:57.020667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ca27fb7a-b66a-4c5a-85e7-b738f9f2b2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:47:57.022206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1338/2629] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:47:56 2022] DELETE /identity/v3/projects/211606cc420342be8deb3eb71c296ed9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:47:57.028618 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.028974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.029317 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.047612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:57.048924 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:57.107667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1292/2630] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:47:57.114918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.115213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.115494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.133854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:57.135780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:57.143587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1339/2631] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:57.147798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.158068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66beac3-d5ba-44e8-b168-136821ddad8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.162690 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.163344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.163628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.163946 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.164794 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:47:57.165146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.166768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.179001 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1293/2632] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:47:57.182414 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.191614 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dfc1e-8d70-422a-9c32-8f537c9e7490 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.196447 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.197128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.197919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.198211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.198952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.200948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.220197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:57.228936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1340/2633] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:47:57 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:47:57.232983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.242967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357392f6-6bc4-49cb-b614-2735819d17a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.304067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.304769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.305030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.305310 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.306005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.308059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.328075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1294/2634] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:47:57 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:47:57.332268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.343013 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eecb15-0e58-43c8-995a-e5aff3c11bf3 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.347588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.348277 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.348563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.348839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.349567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.351153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.361596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1341/2635] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:47:57.365267 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.371950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:57.376032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d6a72-92a0-41b0-a2ab-c956ef626791 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.381328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.381982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.382268 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.382545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.383266 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.385365 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5fb9b-45da-493e-9fe6-e1a15268a6e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.395142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1295/2636] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:47:57.399108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.409429 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba67f16-60a0-459f-ba33-c4af040d025f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.413908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.414795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.415074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.415346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] PATH_INFO: `/v3/projects/3c0c722ba1bf46d8b1c1ae938190ce64/users/c0febff5a5864702bed6ecaeb70bbfb7/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.431224 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0c722ba1bf46d8b1c1ae938190ce64, user_id=c0febff5a5864702bed6ecaeb70bbfb7, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.433859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.448469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1342/2637] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:47:57 2022] PUT /identity/v3/projects/3c0c722ba1bf46d8b1c1ae938190ce64/users/c0febff5a5864702bed6ecaeb70bbfb7/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:47:57.458676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.458990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.459284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.485547 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] MFA Rules not processed for user `c0febff5a5864702bed6ecaeb70bbfb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:47:57.486737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:57.548523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1296/2638] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:47:57.560460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:57.562191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:57.621298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95cd1-67cf-4e57-a8e2-37c7fcb38ef8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:57.624930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:57.625830 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:57.626115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:57.626392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:57.634730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:57.636196 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83966d1-ce7f-4f20-9936-d6682269e6b8 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:57.644984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1343/2639] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:07.545629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=35 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:07.640952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:25.135029 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:25.136475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:25.221186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce47862-5634-4274-9e94-1cbd08fd15fc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:25.223454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:25.224159 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:25.224442 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:25.224716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:25.226504 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:25.239703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:25.241293 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:25.252247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1297/2640] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:30.598648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:30.599110 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:30.599332 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:30.670000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:30.671229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:30.741694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1344/2641] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:30.798132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:30.800637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:30.935045 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71df5-eb6d-4f30-91cb-c5daaae2884c service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:30.940925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:30.941926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:30.941926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:30.942139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:30.950949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:30.952585 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:30.962670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1298/2642] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:48:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:48:34.902929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:34.904340 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:34.961167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e39b3-af6b-4a46-bc4d-fce2ed7fd345 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:34.964928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:34.965765 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:34.966050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:34.966330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:34.968134 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:35.034138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:35.035612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:35.046229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1345/2643] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:38.879481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:38.881570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:38.896607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42194e6-78b3-40c0-b6c8-9569051f42f6 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:38.901543 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:38.902426 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:38.902712 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:38.902992 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] PATH_INFO: `/v3/users/dac164d6b76a4030ac4ebce2675dc5ad` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:38.904247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac164d6b76a4030ac4ebce2675dc5ad)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:38.906699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:38.958966 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac164d6b76a4030ac4ebce2675dc5ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:38.959625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac164d6b76a4030ac4ebce2675dc5ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:38.961643 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:38.978720 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac164d6b76a4030ac4ebce2675dc5ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:38.987328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac164d6b76a4030ac4ebce2675dc5ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:38.988324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-97a14993-946f-4e77-b607-edb1ccc5ea66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac164d6b76a4030ac4ebce2675dc5ad'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.007461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1299/2644] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:38 2022] DELETE /identity/v3/users/dac164d6b76a4030ac4ebce2675dc5ad => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:39.207520 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.220561 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ed5b3-0fc4-4bf9-987f-0c384a11fe3c service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.245117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:39.297767 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.298743 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.299361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.299889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] PATH_INFO: `/v3/projects/abad8e0d4081462bad168cbb61f83a01` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.301627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abad8e0d4081462bad168cbb61f83a01)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.304034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.359894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abad8e0d4081462bad168cbb61f83a01', 'request_id': 'req-d79e8f23-fb96-40c7-9808-3ecdc5f71173'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.366284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abad8e0d4081462bad168cbb61f83a01', 'request_id': 'req-d79e8f23-fb96-40c7-9808-3ecdc5f71173'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.376536 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abad8e0d4081462bad168cbb61f83a01', 'request_id': 'req-d79e8f23-fb96-40c7-9808-3ecdc5f71173'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.390146 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] The token cache is being invalidate because project abad8e0d4081462bad168cbb61f83a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:39.390754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.391598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-d79e8f23-fb96-40c7-9808-3ecdc5f71173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:39.393576 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1346/2645] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:39 2022] DELETE /identity/v3/projects/abad8e0d4081462bad168cbb61f83a01 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:39.398749 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.399203 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.399550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.418581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:39.420512 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:39.490315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1300/2646] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:39.498648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.498936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.499223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.518402 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:39.519638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:39.527680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1347/2647] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:39.532465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.543044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38edc8-3267-40d8-a190-0912ff22a043 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.549921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.550534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.550823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.551108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.551880 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:48:39.552195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.554058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.572188 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1301/2648] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:48:39.573597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.589194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f275f5-0db2-4f04-9483-9fa9663c6937 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.595880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.596848 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.597452 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.598041 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.609549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.611469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.647814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1348/2649] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:48:39 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:39.649912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.665103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd7913-34e4-4266-9437-925d80286bcd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.729008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.729685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.729970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.730244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.730932 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.732911 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.759660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1302/2650] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:48:39 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:39.760518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.796512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216573d-6121-40a7-9ac8-c7b27c14f409 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.801976 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.802584 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.802907 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.803150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.803973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.805569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.818139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1349/2651] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:48:39.821877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.832704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89d064-0d9f-4e32-b474-32592ba4a2f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.837725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.838397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.838681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.839007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.839942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.841548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a194ccd-5afc-4cff-ab03-c986b157c34a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.852152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1303/2652] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:39.855745 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:39.866839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509591-b6ff-4c8e-8979-0da6371324e9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:39.871821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:39.872703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.872980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.873851 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] PATH_INFO: `/v3/projects/fc9f83d7d96e4034b5ba6babfbdeece8/users/824b49494d05452a8fc628f6eee18a27/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.889929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9f83d7d96e4034b5ba6babfbdeece8, user_id=824b49494d05452a8fc628f6eee18a27, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:39.892695 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:39.907012 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1350/2653] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:39 2022] PUT /identity/v3/projects/fc9f83d7d96e4034b5ba6babfbdeece8/users/824b49494d05452a8fc628f6eee18a27/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:39.917441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:39.917987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:39.918111 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:39.948150 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] MFA Rules not processed for user `824b49494d05452a8fc628f6eee18a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:39.949389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:39.973104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:40.025778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1304/2654] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:40.036760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:40.039542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:40.100785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:40.128107 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9ef19-422d-4847-bfdb-3bada08b968e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:40.133661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:40.134365 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:40.134650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:40.134925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:40.143055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:40.144676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89076e-a88a-4675-b7ce-1e1970b3a434 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:40.153593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1351/2655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:42.994811 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:43.081930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618fd29-990d-41f1-b487-8b0549854734 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:43.088053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:43.089069 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:43.089583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:43.089935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] PATH_INFO: `/v3/users/824b49494d05452a8fc628f6eee18a27` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:43.091579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=824b49494d05452a8fc628f6eee18a27)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:43.095928 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:43.179424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824b49494d05452a8fc628f6eee18a27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.180156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824b49494d05452a8fc628f6eee18a27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.194145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824b49494d05452a8fc628f6eee18a27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.201411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824b49494d05452a8fc628f6eee18a27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.202699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824b49494d05452a8fc628f6eee18a27'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.219877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1305/2656] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:42 2022] DELETE /identity/v3/users/824b49494d05452a8fc628f6eee18a27 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:43.223420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:43.223941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:43.229349 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:43.252296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:43.253717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:43.333434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1352/2657] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:43.344594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:43.346352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:43.417175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:43.448311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10260dd-8455-4551-ac5c-d5a830620f36 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:43.454198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:43.454935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:43.455292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:43.455684 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:43.469670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:43.471309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9f846-e2dd-4585-b2dc-94041838b955 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:43.491338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1306/2658] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:43.800441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:43.801885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:43.815822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba4510-d580-498c-9107-997b7913ebd3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:43.821182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:43.822189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:43.822620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:43.822967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] PATH_INFO: `/v3/projects/fc9f83d7d96e4034b5ba6babfbdeece8` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:43.824367 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9f83d7d96e4034b5ba6babfbdeece8)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:43.826601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:43.899857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9f83d7d96e4034b5ba6babfbdeece8', 'request_id': 'req-4202b3a7-01d2-4e21-9430-4775df10613b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.902449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9f83d7d96e4034b5ba6babfbdeece8', 'request_id': 'req-4202b3a7-01d2-4e21-9430-4775df10613b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.916449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9f83d7d96e4034b5ba6babfbdeece8', 'request_id': 'req-4202b3a7-01d2-4e21-9430-4775df10613b'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.934108 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] The token cache is being invalidate because project fc9f83d7d96e4034b5ba6babfbdeece8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:43.934624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.935426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4202b3a7-01d2-4e21-9430-4775df10613b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:43.937435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1353/2659] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:43 2022] DELETE /identity/v3/projects/fc9f83d7d96e4034b5ba6babfbdeece8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:43.942122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:43.942638 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:43.942906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:43.966604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:43.968636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:44.041566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1307/2660] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:48:44.050442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.051045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.051530 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.071529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:44.073024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:44.081710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1354/2661] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:44.086165 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.097468 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e240a90-e98d-47ec-8953-90a107c757b2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.103675 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.104895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.105249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.105592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.106411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:48:44.106783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.109067 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.128481 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1308/2662] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:48:44.129646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.140207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfa069-8ffb-47bb-8a2c-706801f0c021 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.145474 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.146162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.146441 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.146717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.147396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.149333 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.183661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1355/2663] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:44.184935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.199264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559ab63-f619-41e5-a27f-5dc52c820672 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.279326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.280644 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.281006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.281247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.281929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.283970 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.306474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1309/2664] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:44.311280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.327512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9c52-e3b6-4f06-8c6b-025f35a52a34 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.335581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.336446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.336738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.337686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.338468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.340147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.351807 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1356/2665] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:44.355959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.366635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a7a65-1979-4c62-8b77-767caaa408df admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.371363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.372052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.372335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.372607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.373328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.375785 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09baac-2fda-4ab9-80e7-ed58ac52117a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.386064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1310/2666] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:44.390483 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.400617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216b825-bebd-40c1-8c42-da9d46431751 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.407566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.408583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.408953 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.409293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] PATH_INFO: `/v3/projects/9fcba35d4a92409797a211fb4a2e71ce/users/cea6acedc5ec495cb195e4cbc848e95e/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.425658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcba35d4a92409797a211fb4a2e71ce, user_id=cea6acedc5ec495cb195e4cbc848e95e, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.428384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.430880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:44.448059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1357/2667] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:44 2022] PUT /identity/v3/projects/9fcba35d4a92409797a211fb4a2e71ce/users/cea6acedc5ec495cb195e4cbc848e95e/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:48:44.455707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.456032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.456309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.484110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] MFA Rules not processed for user `cea6acedc5ec495cb195e4cbc848e95e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:44.485323 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:44.536182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:44.567553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1311/2668] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:44.572055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.647396 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9642c7-2aa4-470a-97e2-d81db0e011be admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.652622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.653159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.653455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.653735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.654473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.657247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.689937 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1358/2669] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:48:44.691060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.702732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c6245-a611-4684-9111-d2429e798f02 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.767514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.768284 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.768567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.768850 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.769570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.771605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.796676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1312/2670] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:48:44 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:48:44.797815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.808508 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dedfe2-ce00-4fb7-bfbc-e4f452677dfa admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.813174 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.813859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.814159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.814439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.815191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.816832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.829249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1359/2671] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:48:44.833685 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.846196 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210bc2f-ea26-47ca-b3eb-8c70a9b2acdc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.853272 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.854294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.854681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.855108 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] PATH_INFO: `/v3/projects/2e3c03f555e34d74ad81b01154cebf41/users/11b82af849e64175b4e9e1c06fc67ff5/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.872681 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3c03f555e34d74ad81b01154cebf41, user_id=11b82af849e64175b4e9e1c06fc67ff5, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.875860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:44.896106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1313/2672] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:44 2022] PUT /identity/v3/projects/2e3c03f555e34d74ad81b01154cebf41/users/11b82af849e64175b4e9e1c06fc67ff5/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:48:44.901688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:44.914790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1e423-0cfa-454b-91ac-bb430afc6d6d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:44.992077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:44.992841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:44.993131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:44.993410 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:44.994173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:44.995890 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194500a-22e9-480e-a761-1a134b00c1d0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:45.008099 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1360/2673] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:45.012829 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:45.025275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185a05c-6276-4dee-ae0d-88f897fde390 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:45.031663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:45.032886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:45.033270 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:45.033725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] PATH_INFO: `/v3/projects/2e3c03f555e34d74ad81b01154cebf41/users/11b82af849e64175b4e9e1c06fc67ff5/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:45.035872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3c03f555e34d74ad81b01154cebf41, user_id=11b82af849e64175b4e9e1c06fc67ff5, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:45.038761 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:45.055350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1314/2674] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:45 2022] PUT /identity/v3/projects/2e3c03f555e34d74ad81b01154cebf41/users/11b82af849e64175b4e9e1c06fc67ff5/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:45.065527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:45.065818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:45.066097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:45.098784 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] MFA Rules not processed for user `11b82af849e64175b4e9e1c06fc67ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:45.100186 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:45.162824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:45.188738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1361/2675] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:45.203813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:45.205682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:45.282128 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f64ea-cf58-49c7-886a-78a8e0d6c442 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:45.286027 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:45.286694 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:45.286986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:45.287260 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:45.298009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:45.299455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:45.320445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1315/2676] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:45.409272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:45.411215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:45.435540 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef5e13-8e40-493d-93c2-56e6812327d3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:45.440320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:45.441278 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:45.441723 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:45.442176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:45.540833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:45.542339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4abb4-770f-47b0-a999-f0f0dd88f90d service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:45.571478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1362/2677] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:46.652133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:46.653569 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:46.697898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:46.725193 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1202-7664-4ee8-89f1-6735acefc158 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:46.730282 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:46.731172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:46.731454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:46.731729 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] PATH_INFO: `/v3/users/c0febff5a5864702bed6ecaeb70bbfb7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:46.733116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0febff5a5864702bed6ecaeb70bbfb7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:46.735056 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:46.798994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0febff5a5864702bed6ecaeb70bbfb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:46.799658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0febff5a5864702bed6ecaeb70bbfb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:46.817152 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0febff5a5864702bed6ecaeb70bbfb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:46.825136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0febff5a5864702bed6ecaeb70bbfb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:46.825990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0febff5a5864702bed6ecaeb70bbfb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:46.847105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1316/2678] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:46 2022] DELETE /identity/v3/users/c0febff5a5864702bed6ecaeb70bbfb7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:46.852566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:46.853138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:46.853138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:46.890140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:46.891336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:46.973795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1363/2679] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:48:46.984176 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:46.986755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:47.068857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6df5c-f9cb-43ff-8097-34d7507ab3ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:47.074212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:47.075042 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:47.075306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:47.075603 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:47.088092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:47.089819 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5497595-8028-4655-8d73-4159408f45a3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:47.100233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1317/2680] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:47.415581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:47.417042 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:47.429789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2760ae-f7e6-4d3e-ad23-da81961520ef None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:47.437376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:47.438234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:47.438516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:47.438790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] PATH_INFO: `/v3/projects/3c0c722ba1bf46d8b1c1ae938190ce64` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:47.440009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0c722ba1bf46d8b1c1ae938190ce64)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:47.441918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:47.488007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0c722ba1bf46d8b1c1ae938190ce64', 'request_id': 'req-da364e77-909b-454c-aa7f-0abc2c060a82'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:47.493497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0c722ba1bf46d8b1c1ae938190ce64', 'request_id': 'req-da364e77-909b-454c-aa7f-0abc2c060a82'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:47.501269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0c722ba1bf46d8b1c1ae938190ce64', 'request_id': 'req-da364e77-909b-454c-aa7f-0abc2c060a82'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:47.512058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] The token cache is being invalidate because project 3c0c722ba1bf46d8b1c1ae938190ce64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:47.512345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:47.512974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-da364e77-909b-454c-aa7f-0abc2c060a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:47.514751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1364/2681] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:47 2022] DELETE /identity/v3/projects/3c0c722ba1bf46d8b1c1ae938190ce64 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:55.177947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:55.178451 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:55.178869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:55.181254 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1318/2682] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jul 20 15:48:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:48:55.186795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:55.187265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:55.187733 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:55.213453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:55.234431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] MFA Rules not processed for user `e441b994217b4b81be762246ef726545`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:55.236242 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:55.314091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1365/2683] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jul 20 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:55.329120 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:55.330842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:55.399725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:55.419389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df1aa-e733-4233-b62e-f7480626d4e7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:55.423710 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:55.424414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:55.424699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:55.424974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:55.437915 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:55.439385 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:55.451417 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1319/2684] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:55.643655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:55.645295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:55.721982 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467d0dd-833e-4b5c-907d-5cd5ac9aeee9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:55.726959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:55.727604 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:55.727922 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:55.728194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:55.729952 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:55.741276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:55.742841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc69894-f689-4ed7-886d-aa682f4f89b8 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:55.754300 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1366/2685] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:56.999312 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.002074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.068591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548b0ee-8961-4095-a453-4c9e23e82b14 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.073311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.074213 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.074500 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.074774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] PATH_INFO: `/v3/users/cea6acedc5ec495cb195e4cbc848e95e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.076005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea6acedc5ec495cb195e4cbc848e95e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.077959 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.134561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea6acedc5ec495cb195e4cbc848e95e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.136955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea6acedc5ec495cb195e4cbc848e95e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.149379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea6acedc5ec495cb195e4cbc848e95e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.159304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea6acedc5ec495cb195e4cbc848e95e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.160407 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea6acedc5ec495cb195e4cbc848e95e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.175573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1320/2686] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:56 2022] DELETE /identity/v3/users/cea6acedc5ec495cb195e4cbc848e95e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:57.180130 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec76f-0229-4259-b390-d81abd6ccdae None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.180714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec76f-0229-4259-b390-d81abd6ccdae None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.180714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec76f-0229-4259-b390-d81abd6ccdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.198445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-d22ec76f-0229-4259-b390-d81abd6ccdae None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:57.199646 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d22ec76f-0229-4259-b390-d81abd6ccdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.261828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1367/2687] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:48:57.273304 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.274877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.334842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61601-a72d-45f2-ab23-8098d88c4bf0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.338561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.339151 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.339434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.339711 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.348955 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.350363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.360227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1321/2688] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:57.620913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.621205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.621484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.639573 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:57.640815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.654824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:57.709050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1368/2689] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jul 20 15:48:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:57.720913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.722309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.728597 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.730051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.731553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:57.748605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f039a-fd1c-4c07-9d67-73aa7085c0b4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.755315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.756495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.756850 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.757253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] PATH_INFO: `/v3/projects/9fcba35d4a92409797a211fb4a2e71ce` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.758702 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcba35d4a92409797a211fb4a2e71ce)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.761218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.809382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcba35d4a92409797a211fb4a2e71ce', 'request_id': 'req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.816482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcba35d4a92409797a211fb4a2e71ce', 'request_id': 'req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.825100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8d973-dd18-4269-b0dd-3c3b440d8269 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.825856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcba35d4a92409797a211fb4a2e71ce', 'request_id': 'req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.837253 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] The token cache is being invalidate because project 9fcba35d4a92409797a211fb4a2e71ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:57.837585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.838201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.840049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1369/2690] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:57 2022] DELETE /identity/v3/projects/9fcba35d4a92409797a211fb4a2e71ce => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:48:57.843694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.845046 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.888919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.889673 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.889977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.890458 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.892550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.913574 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a0173-55fc-43fd-b7b3-a2dbd392b490 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.918590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.919467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.919751 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.920058 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] PATH_INFO: `/v3/users/11b82af849e64175b4e9e1c06fc67ff5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.921268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b82af849e64175b4e9e1c06fc67ff5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.923828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.954539 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.956189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.964541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1322/2691] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:48:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:48:57.973031 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b82af849e64175b4e9e1c06fc67ff5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.975681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:57.987714 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b82af849e64175b4e9e1c06fc67ff5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.988694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b82af849e64175b4e9e1c06fc67ff5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.989606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b82af849e64175b4e9e1c06fc67ff5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.996758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-675cbc05-0166-4ada-b45f-0c179b13a03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b82af849e64175b4e9e1c06fc67ff5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.029996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1370/2692] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:57 2022] DELETE /identity/v3/users/11b82af849e64175b4e9e1c06fc67ff5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:48:58.426906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:58.455471 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b97a-06da-4cf9-b378-1b01eb3aa6c5 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:58.524562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:58.562315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:58.563564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.564279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.564646 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] PATH_INFO: `/v3/projects/2e3c03f555e34d74ad81b01154cebf41` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.566162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3c03f555e34d74ad81b01154cebf41)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:58.568594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:58.645783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3c03f555e34d74ad81b01154cebf41', 'request_id': 'req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.653262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3c03f555e34d74ad81b01154cebf41', 'request_id': 'req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.660755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3c03f555e34d74ad81b01154cebf41', 'request_id': 'req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.679015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] The token cache is being invalidate because project 2e3c03f555e34d74ad81b01154cebf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:58.679409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.680162 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7f8e7794-4c59-4257-9d9a-53ba5f9c159d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.681768 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1323/2693] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:58 2022] DELETE /identity/v3/projects/2e3c03f555e34d74ad81b01154cebf41 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:58.686820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.687353 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.687527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.706587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:58.707944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:58.798050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1371/2694] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:58.805613 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.806232 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.806780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.827843 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:58.829271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:58.841015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1324/2695] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:58.842260 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:58.858941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1d4f3-f0da-4bab-a456-f0b2b473c21f None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:58.867317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:58.867814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.868320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.868619 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.869455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:48:58.869774 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:58.873115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:58.889377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1372/2696] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:48:58.890908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:58.905492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6f479-8fee-4b16-81c5-53229a3a6f47 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:58.910906 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:58.911596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.911905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.912814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.912940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:58.914810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:58.962411 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1325/2697] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:48:58 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:58.966348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:58.977560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eb7dc-d9c9-4b84-b782-d5e53a9ebec0 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.045817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.046533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.046814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.047086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.048362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.050261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.072109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1373/2698] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:48:58 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:59.076658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.090611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89000-ba99-42ba-8b7b-a66424959e73 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.096023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.096769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.097116 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.097452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.098246 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.099981 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.111935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1326/2699] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:48:59.116136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.127106 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3155c37-be27-4aac-ba2b-d6135cb45f20 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.132139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.132924 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.133207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.133612 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.134532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.136337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102df8b-84a5-4dbe-a87e-e430c17804bf admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.148388 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1374/2700] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:59.152578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.163971 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4809e-27aa-4ec1-92d6-5db9d5fe68ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.168802 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.169820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.170180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.171860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] PATH_INFO: `/v3/projects/51355878be034160b4753c75012700f3/users/f8a3a51c44e845ad91c38845558b4dcc/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.187253 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51355878be034160b4753c75012700f3, user_id=f8a3a51c44e845ad91c38845558b4dcc, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.190057 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.206130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1327/2701] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:59 2022] PUT /identity/v3/projects/51355878be034160b4753c75012700f3/users/f8a3a51c44e845ad91c38845558b4dcc/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:59.216718 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.216931 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.217210 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.247537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] MFA Rules not processed for user `f8a3a51c44e845ad91c38845558b4dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:59.248836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:59.271857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:59.341652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1375/2702] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:59.342752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.418306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3c8f2-821a-4858-9992-b9d0c84252e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.432415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.433177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.433478 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.433740 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.434462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.436514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.458036 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:59.467908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1328/2703] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:48:59 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:48:59.473065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.488820 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30627bc8-4264-4061-b49d-fa22be7c9bc8 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.550173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.550857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.551142 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.551422 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.552518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.554478 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.577525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1376/2704] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:48:59 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:48:59.582156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.593320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4d02-fbec-425c-84e0-a0f9bad35e08 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.598544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.599222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.599509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.599815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.600560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.602353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.622125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1329/2705] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:48:59.626969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.637775 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8bed7-b31a-409c-af6c-3b16d58f014b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.646466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.647411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.647854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.647977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] PATH_INFO: `/v3/projects/e938a64d506c493faf06a2b6db9e1ee8/users/bb86658992d246598d964bb56c4eda15/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.664951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e938a64d506c493faf06a2b6db9e1ee8, user_id=bb86658992d246598d964bb56c4eda15, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.667622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.685449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1377/2706] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:59 2022] PUT /identity/v3/projects/e938a64d506c493faf06a2b6db9e1ee8/users/bb86658992d246598d964bb56c4eda15/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:59.690104 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.705127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb332be-9445-4b8a-b614-321f0c7746ce admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.767006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.767582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.767888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.771738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.772653 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.774379 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f69a3-4bab-4144-af69-f291d8fcddea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.787806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1330/2707] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:48:59.790541 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.801631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d4819-d4cd-4c63-934f-27b8d480e083 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.806459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.807423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.807798 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.808159 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] PATH_INFO: `/v3/projects/e938a64d506c493faf06a2b6db9e1ee8/users/bb86658992d246598d964bb56c4eda15/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.810408 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e938a64d506c493faf06a2b6db9e1ee8, user_id=bb86658992d246598d964bb56c4eda15, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.814704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.825779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1378/2708] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:48:59 2022] PUT /identity/v3/projects/e938a64d506c493faf06a2b6db9e1ee8/users/bb86658992d246598d964bb56c4eda15/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:59.837741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.838061 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.838356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.866655 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] MFA Rules not processed for user `bb86658992d246598d964bb56c4eda15`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:48:59.868186 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:59.947281 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1331/2709] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:48:59.961760 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.963753 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:00.030491 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bd58-f172-4293-b9e7-803e6dba0de9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:00.044588 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:00.045911 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:00.046221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:00.046501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:00.057835 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:00.059287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:00.073567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1379/2710] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:00.138218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:00.143014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:00.172300 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40a0de-8793-413e-b66e-a72fdae5d0fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:00.185796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:00.186476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:00.186763 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:00.187040 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:00.262037 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:00.263878 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45dd87-ae27-4585-8c28-196668f77005 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:00.277275 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1332/2711] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:03.824857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:03.844122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:03.895923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8161b2-26a6-4cbb-9934-f4de450f615f service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:03.900735 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:03.901628 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:03.901913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:03.902193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] PATH_INFO: `/v3/users/f8a3a51c44e845ad91c38845558b4dcc` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:03.903415 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a3a51c44e845ad91c38845558b4dcc)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:03.905417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:03.967389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a3a51c44e845ad91c38845558b4dcc'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:03.981037 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a3a51c44e845ad91c38845558b4dcc'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:03.981769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a3a51c44e845ad91c38845558b4dcc'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:03.982377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a3a51c44e845ad91c38845558b4dcc'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:03.990502 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a3a51c44e845ad91c38845558b4dcc'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.007261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1380/2712] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:03 2022] DELETE /identity/v3/users/f8a3a51c44e845ad91c38845558b4dcc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:04.011582 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:04.012149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:04.012364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:04.025173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:04.035032 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:04.036378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:04.107350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1333/2713] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:04.121709 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:04.123411 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:04.192226 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3796fc-d5f7-478a-b9f2-a8b1e5c1042e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:04.195928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6516-8309-423b-9198-0714250a40af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:04.196587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6516-8309-423b-9198-0714250a40af service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:04.196868 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6516-8309-423b-9198-0714250a40af service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:04.197141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6516-8309-423b-9198-0714250a40af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:04.207941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6516-8309-423b-9198-0714250a40af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:04.210131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6516-8309-423b-9198-0714250a40af service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:04.221748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1381/2714] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:04.553717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:04.555366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:04.569262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5823aa-de0f-4158-8dc1-4f1d10052e4b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:04.574666 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:04.575577 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:04.575903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:04.576222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] PATH_INFO: `/v3/projects/51355878be034160b4753c75012700f3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:04.577446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51355878be034160b4753c75012700f3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:04.579505 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:04.647025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51355878be034160b4753c75012700f3', 'request_id': 'req-b945c447-8d64-455f-ac6f-e417acf8a4c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.653949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51355878be034160b4753c75012700f3', 'request_id': 'req-b945c447-8d64-455f-ac6f-e417acf8a4c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.663021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51355878be034160b4753c75012700f3', 'request_id': 'req-b945c447-8d64-455f-ac6f-e417acf8a4c7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.676985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] The token cache is being invalidate because project 51355878be034160b4753c75012700f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:04.677278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.678004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.679747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1334/2715] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:04 2022] DELETE /identity/v3/projects/51355878be034160b4753c75012700f3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:04.683858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6516-8309-423b-9198-0714250a40af service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:04.697809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6516-8309-423b-9198-0714250a40af service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:04.746758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:04.770291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:04.771563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:04.771967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:04.772249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] PATH_INFO: `/v3/users/bb86658992d246598d964bb56c4eda15` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:04.773459 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb86658992d246598d964bb56c4eda15)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:04.775390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:04.842773 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb86658992d246598d964bb56c4eda15'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.864307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb86658992d246598d964bb56c4eda15'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.865207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb86658992d246598d964bb56c4eda15'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.865837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb86658992d246598d964bb56c4eda15'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.880737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1831db05-622c-4829-bd77-9c5df87757e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb86658992d246598d964bb56c4eda15'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:04.905139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1382/2716] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:04 2022] DELETE /identity/v3/users/bb86658992d246598d964bb56c4eda15 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:49:05.258972 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.260882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:05.279721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945c447-8d64-455f-ac6f-e417acf8a4c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.379321 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.380974 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.381483 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.381983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] PATH_INFO: `/v3/projects/e938a64d506c493faf06a2b6db9e1ee8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.384226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e938a64d506c493faf06a2b6db9e1ee8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.386686 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.433259 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:05.446454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e938a64d506c493faf06a2b6db9e1ee8', 'request_id': 'req-e75d43ad-5bbb-4311-89b6-ac02374410ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:05.455363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e938a64d506c493faf06a2b6db9e1ee8', 'request_id': 'req-e75d43ad-5bbb-4311-89b6-ac02374410ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:05.462890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e938a64d506c493faf06a2b6db9e1ee8', 'request_id': 'req-e75d43ad-5bbb-4311-89b6-ac02374410ec'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:05.476881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] The token cache is being invalidate because project e938a64d506c493faf06a2b6db9e1ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:05.476881 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:05.477783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e75d43ad-5bbb-4311-89b6-ac02374410ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:05.479891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1335/2717] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:05 2022] DELETE /identity/v3/projects/e938a64d506c493faf06a2b6db9e1ee8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:05.484737 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.485109 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.485460 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.504023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:05.505337 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:05.570978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1383/2718] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:05.579289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.579570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.579903 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.598760 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:05.599989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:05.607876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1336/2719] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:05.612182 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.621843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebff025-319b-44b2-8e1d-496bb36a0d05 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.627149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.627818 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.628098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.628371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.629089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:49:05.629392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.630990 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.643052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1384/2720] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:49:05.647492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.657616 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396072-e371-4d9e-99c3-9acee8f3073f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.662566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.663219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.663501 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.663805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.664524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.666399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.693664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1337/2721] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:49:05 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:05.698417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.710009 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb9e9-ba67-42c7-a07b-24c799590446 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.753763 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:05.768622 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.769298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.769581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.769869 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.770562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.772499 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.792239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1385/2722] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:49:05 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:49:05.796378 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.806560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25496e-0f32-4bf3-aa99-215969085cce admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.811269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.811898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.812182 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.812454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.813750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.815328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.825599 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1338/2723] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:05.829963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.839867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1024ff-36e6-4a8e-9e29-1c1b0f0d3822 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.844663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.845321 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.845618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.845895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.846667 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.848363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18ffc-da89-40ac-8b30-9e9928ae4a42 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.858403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1386/2724] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:49:05.862531 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:05.872463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd04a3c-7fd6-4565-87de-203f109d5a70 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:05.877854 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:05.878721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.879005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.879279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] PATH_INFO: `/v3/projects/51fdc5aff535457d8c038015dce3def0/users/73e6721f56f44fde84c8da37f81de971/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.894031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51fdc5aff535457d8c038015dce3def0, user_id=73e6721f56f44fde84c8da37f81de971, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:05.896704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:05.909541 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1339/2725] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:49:05 2022] PUT /identity/v3/projects/51fdc5aff535457d8c038015dce3def0/users/73e6721f56f44fde84c8da37f81de971/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:49:05.919919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:05.920214 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:05.920495 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:05.947871 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] MFA Rules not processed for user `73e6721f56f44fde84c8da37f81de971`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:05.949032 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:06.013606 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1387/2726] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:06.018031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.083882 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad13510-5359-4929-b17b-c6d2299c25f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.088773 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.089470 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.089755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.090034 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.090769 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.092723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.119610 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1340/2727] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:49:06 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:06.123721 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.135051 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fb619-1b06-430f-a1f2-74e785139ffc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.196211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.196891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.197177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.197461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.198164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.200100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.220380 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1388/2728] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:49:06 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:06.224454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.234728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0d54-2cbe-44cd-98f2-a99b8177e65b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.240122 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.240796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.241077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.241353 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.242105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.243715 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.255467 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1341/2729] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:06.259861 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.269897 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d3160-8615-4de9-8123-66d8b2deb7e8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.274419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.275286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.275568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.275881 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] PATH_INFO: `/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8/users/c8e8d256fc97475799489040f72d0763/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.290314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15dc01cb8ac40ceb0ac3ca929a0f7d8, user_id=c8e8d256fc97475799489040f72d0763, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.293070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.306960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1389/2730] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:49:06 2022] PUT /identity/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8/users/c8e8d256fc97475799489040f72d0763/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:06.310885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.322820 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503854d9-96cb-4901-9bf0-5502f7d66ada admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.351538 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:06.380866 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.381530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.381815 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.382087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.382813 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.384431 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f06b9-3e44-46d2-8bea-a38d1fa192ed admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.395222 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1342/2731] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:06.399461 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.410663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8372129-493e-471d-bee4-2c20b8aae6eb admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.415331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.416342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.416620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.416889 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] PATH_INFO: `/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8/users/c8e8d256fc97475799489040f72d0763/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.418856 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15dc01cb8ac40ceb0ac3ca929a0f7d8, user_id=c8e8d256fc97475799489040f72d0763, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.421534 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.433276 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1390/2732] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:49:06 2022] PUT /identity/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8/users/c8e8d256fc97475799489040f72d0763/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:06.444329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.444622 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.444899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.471721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] MFA Rules not processed for user `c8e8d256fc97475799489040f72d0763`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:06.472969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:06.540263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1343/2733] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:06.551389 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.553074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:06.562827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:06.611546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315ba45-b093-43d2-95da-72c326d28c8e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.615138 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.615791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.616086 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.616363 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.625268 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.626983 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.636351 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1391/2734] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:06.784086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:06.793799 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1cba4-598d-46ca-9778-1f1e924147a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:06.797297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:06.797933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:06.798216 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:06.798629 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:06.854448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:06.855897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c7fcf-5056-40a1-8d75-2bfb3032d96a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:06.864696 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1344/2735] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:16.167067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:16.233297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97045307-dc85-4d00-a608-4423e7cf9060 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:16.238641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:16.239514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:16.239936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:16.240217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] PATH_INFO: `/v3/users/73e6721f56f44fde84c8da37f81de971` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:16.241398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e6721f56f44fde84c8da37f81de971)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:16.243341 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:16.287542 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e6721f56f44fde84c8da37f81de971'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.300059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e6721f56f44fde84c8da37f81de971'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.300725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e6721f56f44fde84c8da37f81de971'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.301336 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e6721f56f44fde84c8da37f81de971'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.308147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e6721f56f44fde84c8da37f81de971'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.320314 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1392/2736] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:16 2022] DELETE /identity/v3/users/73e6721f56f44fde84c8da37f81de971 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:16.324860 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:16.325328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:16.325454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:16.341855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:16.343030 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:16.387579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:16.402503 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1345/2737] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:16.412514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:16.413927 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:16.446016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:16.474439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcd0ad-a68e-4608-aaa5-8aff5ba775e9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:16.478121 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:16.478776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:16.479063 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:16.479344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:16.488434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:16.489872 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:16.500197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1393/2738] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:16.849494 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:16.850938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:16.864409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739131a-c2e4-461c-a13a-200b27878d91 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:16.869429 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:16.870429 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:16.870879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:16.871227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] PATH_INFO: `/v3/projects/51fdc5aff535457d8c038015dce3def0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:16.872701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51fdc5aff535457d8c038015dce3def0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:16.875695 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:16.921614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51fdc5aff535457d8c038015dce3def0', 'request_id': 'req-356b4a5f-5763-41b9-abe5-14be3db0e952'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.927082 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51fdc5aff535457d8c038015dce3def0', 'request_id': 'req-356b4a5f-5763-41b9-abe5-14be3db0e952'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.933125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51fdc5aff535457d8c038015dce3def0', 'request_id': 'req-356b4a5f-5763-41b9-abe5-14be3db0e952'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.945896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] The token cache is being invalidate because project 51fdc5aff535457d8c038015dce3def0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:16.946183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.946792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:16.948380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1346/2739] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:16 2022] DELETE /identity/v3/projects/51fdc5aff535457d8c038015dce3def0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:16.952345 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:16.962669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967467d-93d9-4a2b-b406-1731bdc9222b service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.020607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.021685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.022027 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.022409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] PATH_INFO: `/v3/users/c8e8d256fc97475799489040f72d0763` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.023833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e8d256fc97475799489040f72d0763)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.025791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.070966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e8d256fc97475799489040f72d0763'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.083081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e8d256fc97475799489040f72d0763'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.083748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e8d256fc97475799489040f72d0763'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.084371 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e8d256fc97475799489040f72d0763'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.091034 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-5da49165-2ffa-400f-8b38-6068385a2160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e8d256fc97475799489040f72d0763'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.104065 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1394/2740] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:16 2022] DELETE /identity/v3/users/c8e8d256fc97475799489040f72d0763 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:17.390611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.392229 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:17.407721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4a5f-5763-41b9-abe5-14be3db0e952 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.478305 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.479262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.479472 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.480010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] PATH_INFO: `/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.481158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15dc01cb8ac40ceb0ac3ca929a0f7d8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.483233 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.533945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15dc01cb8ac40ceb0ac3ca929a0f7d8', 'request_id': 'req-a204eb71-08fd-4018-9cc1-968b2445edde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.540631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15dc01cb8ac40ceb0ac3ca929a0f7d8', 'request_id': 'req-a204eb71-08fd-4018-9cc1-968b2445edde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.547227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15dc01cb8ac40ceb0ac3ca929a0f7d8', 'request_id': 'req-a204eb71-08fd-4018-9cc1-968b2445edde'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.560452 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] The token cache is being invalidate because project d15dc01cb8ac40ceb0ac3ca929a0f7d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:17.560661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.561320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a204eb71-08fd-4018-9cc1-968b2445edde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:17.563526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1347/2741] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:17 2022] DELETE /identity/v3/projects/d15dc01cb8ac40ceb0ac3ca929a0f7d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:49:17.568369 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.568908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.568908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.587693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:17.589024 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:17.635616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:17.652173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1395/2742] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:17.659917 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.660209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.660489 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.669153 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:17.681806 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:17.683019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:17.691741 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1348/2743] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:17.696023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.706555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5529db1-41e0-4ee8-9f2d-6d06da74152c None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.711149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.711854 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.712139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.712412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.713144 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:49:17.713456 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.715045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.726652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1396/2744] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:49:17.730869 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.740548 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30013-d7cd-407d-a573-1c96b27fea1c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.745307 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.745964 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.746249 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.746526 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.747244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.749227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.774796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1349/2745] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:49:17 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:17.778828 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.789047 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4590a4-c309-4c3b-83bc-cffe6fb24f83 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.846074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.846824 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.847111 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.847384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.848139 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.850207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.871399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1397/2746] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:49:17 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:17.876250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.886637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e756-1d49-4175-9cd6-95e26f60eb8c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.893879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.894589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.894872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.895146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.895919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.897519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.908670 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1350/2747] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:17.912590 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.922651 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af752e-4bd3-4f2e-9386-0824c8f3c812 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.927665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.928458 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.928817 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.929164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.929969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.931650 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e566d-d275-4f96-9b91-14899610f7be admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.942923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1398/2748] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:17.947096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:17.957640 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d4cf-8c37-4b9a-b201-fee15c5ddd87 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:17.962324 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:17.963209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:17.963492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:17.963783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] PATH_INFO: `/v3/projects/c76ea6d328944a52b158ab0e654cc51e/users/1e5ef31730d54177b07ae1d3d5bb4ae4/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:17.979517 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76ea6d328944a52b158ab0e654cc51e, user_id=1e5ef31730d54177b07ae1d3d5bb4ae4, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:17.982563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:17.997350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1351/2749] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:49:17 2022] PUT /identity/v3/projects/c76ea6d328944a52b158ab0e654cc51e/users/1e5ef31730d54177b07ae1d3d5bb4ae4/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:49:18.010143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8749b-e4fd-4825-b6cc-a9b16c1766c8 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:18.010439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8749b-e4fd-4825-b6cc-a9b16c1766c8 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:18.010719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8749b-e4fd-4825-b6cc-a9b16c1766c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:18.036969 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-eff8749b-e4fd-4825-b6cc-a9b16c1766c8 None None] MFA Rules not processed for user `1e5ef31730d54177b07ae1d3d5bb4ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:18.038178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-eff8749b-e4fd-4825-b6cc-a9b16c1766c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:18.101661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1399/2750] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:18.112762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:18.114485 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:18.170841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c861-2ebd-4be3-a944-279ff75ed1d1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:18.174942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:18.175579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:18.175888 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:18.176189 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:18.184349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:18.185757 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:18.195169 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1352/2751] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:19.064406 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:19.064913 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:19.065011 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:19.084207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] MFA Rules not processed for user `bd93f75ca8ee4da0b1f1d89ab59b1733`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:19.085398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:19.130518 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:19.145165 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1400/2752] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jul 20 15:49:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:19.157963 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:19.159349 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:19.230006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306b698-8999-4b26-a340-9d1c03fea429 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:19.236119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bd7c157dada45049506a091a97079bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:19.236774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:19.237062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:19.237341 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:19.245594 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:19.247190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017d2a5-9e32-4d61-a272-f99b93d54717 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:19.255533 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1353/2753] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jul 20 15:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:49:21.169164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:21.238236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6f823-04a7-4c82-91b7-e613b5368204 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:21.242689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:21.243550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.243932 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.244204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] PATH_INFO: `/v3/users/1e5ef31730d54177b07ae1d3d5bb4ae4` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.245382 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5ef31730d54177b07ae1d3d5bb4ae4)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:21.247302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:21.293742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5ef31730d54177b07ae1d3d5bb4ae4'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.305686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5ef31730d54177b07ae1d3d5bb4ae4'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.306355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5ef31730d54177b07ae1d3d5bb4ae4'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.306973 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5ef31730d54177b07ae1d3d5bb4ae4'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.314362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5ef31730d54177b07ae1d3d5bb4ae4'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.326758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1401/2754] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:21 2022] DELETE /identity/v3/users/1e5ef31730d54177b07ae1d3d5bb4ae4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:21.331962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.332438 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.332537 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.349129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:21.350316 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:21.379624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:21.411561 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1354/2755] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:49:21.421014 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:21.422512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:21.483579 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798816-a6c3-4da5-ba62-9b88ae73b856 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:21.487194 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:21.487866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.488149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.488423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.497204 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:21.498621 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebef130-d3ae-4348-a1bd-35c1810b2ed3 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:21.508715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1402/2756] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:21.775607 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:21.777163 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:21.794579 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a2d3-0af3-4191-beca-1a05cd00333e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:21.800290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:21.801183 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.801459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.801722 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] PATH_INFO: `/v3/projects/c76ea6d328944a52b158ab0e654cc51e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.803136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76ea6d328944a52b158ab0e654cc51e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:21.805088 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:21.852831 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76ea6d328944a52b158ab0e654cc51e', 'request_id': 'req-521d5175-f7cd-4373-8bea-092da34fca06'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.858735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76ea6d328944a52b158ab0e654cc51e', 'request_id': 'req-521d5175-f7cd-4373-8bea-092da34fca06'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.865335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76ea6d328944a52b158ab0e654cc51e', 'request_id': 'req-521d5175-f7cd-4373-8bea-092da34fca06'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.878925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] The token cache is being invalidate because project c76ea6d328944a52b158ab0e654cc51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:21.879209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.879871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-521d5175-f7cd-4373-8bea-092da34fca06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:21.881440 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1355/2757] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:21 2022] DELETE /identity/v3/projects/c76ea6d328944a52b158ab0e654cc51e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:21.887372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.887656 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.887965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.905521 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:21.906698 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:21.914840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:21.968537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1403/2758] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:21.976413 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:21.976735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:21.977018 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:21.995841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:21.997075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:22.006127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1356/2759] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:22.010394 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.020131 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a4d5-8744-4240-b56d-039853bd1937 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.024754 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.025409 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.025690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.025966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.026694 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:49:22.027003 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.028648 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.040190 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1404/2760] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:49:22.044339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.054078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09d943-e564-4d6e-8f37-dc4e5b3d541b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.058832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.059479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.059789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.060072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.060778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.063285 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.087567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1357/2761] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:49:22 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:49:22.091739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.102136 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae1e11-434a-4005-9c17-58ff13e16f1e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.281899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.282553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.282756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.283035 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.283725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.285641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.304844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1405/2762] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:49:22 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:49:22.309092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.319380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff181-1669-4b5a-8cd0-96106e8dc5e7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.323919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.324580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.324864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.325140 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.325864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.327446 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.337676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1358/2763] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:22.341472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.351675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373f0b-31d1-4cb5-a99c-48f8e6688fa2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.356181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.356822 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.357104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.357377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.358097 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.360317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b418cbf-b4ab-4147-a788-4d13c7797db5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.370061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1406/2764] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:49:22.373995 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.384168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e36e-b2b4-4a49-b1ba-e5687499fe41 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.388676 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.389540 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.389833 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.390110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] PATH_INFO: `/v3/projects/d03174c91dbe422a9fe0befb4c79ff2c/users/0e1efaa71bca49c3a444ffe563d9a019/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.404351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03174c91dbe422a9fe0befb4c79ff2c, user_id=0e1efaa71bca49c3a444ffe563d9a019, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.407492 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.419926 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1359/2765] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:49:22 2022] PUT /identity/v3/projects/d03174c91dbe422a9fe0befb4c79ff2c/users/0e1efaa71bca49c3a444ffe563d9a019/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:22.430948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.431282 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.431564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.457878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] MFA Rules not processed for user `0e1efaa71bca49c3a444ffe563d9a019`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:22.459069 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:22.521439 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1407/2766] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:22.533424 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.535060 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:22.590864 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0468a02-bd37-4191-b441-7d7d3192f277 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.595021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.595663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.595977 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.596258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.602491 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:22.604925 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.606664 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be3b6b-3012-4e90-a983-01ddb2419c08 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.617397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1360/2767] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:22.641806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.651296 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:22.704599 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476819e8-2d7a-4224-94c2-456af3fa024a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.709153 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.710006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.710286 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.710558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] PATH_INFO: `/v3/users/0e1efaa71bca49c3a444ffe563d9a019` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.711786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1efaa71bca49c3a444ffe563d9a019)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.713700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.758213 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1efaa71bca49c3a444ffe563d9a019'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:22.770587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1efaa71bca49c3a444ffe563d9a019'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:22.771272 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1efaa71bca49c3a444ffe563d9a019'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:22.771925 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1efaa71bca49c3a444ffe563d9a019'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:22.778682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1efaa71bca49c3a444ffe563d9a019'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:22.791578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1408/2768] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:49:22 2022] DELETE /identity/v3/users/0e1efaa71bca49c3a444ffe563d9a019 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:22.796166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.796454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.796731 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.813957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:22.815207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:22.872330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1361/2769] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:49:22.882808 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:22.884203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:22.939594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcc160-4546-4c5f-bf14-888babef30da admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:22.943163 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:22.943821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:22.944100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:22.944372 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:22.953227 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:22.955137 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:22.964644 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1409/2770] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:23.478389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:23.480000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:23.492449 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08ec35-3e2b-4b30-a3dc-dcdcd0008bab None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:23.496993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:23.497857 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:23.498133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:23.498400 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] PATH_INFO: `/v3/projects/d03174c91dbe422a9fe0befb4c79ff2c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:23.499574 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03174c91dbe422a9fe0befb4c79ff2c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:23.502374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:23.553897 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03174c91dbe422a9fe0befb4c79ff2c', 'request_id': 'req-c957846b-8d45-41a9-b13b-f6314ea40a0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:23.559329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03174c91dbe422a9fe0befb4c79ff2c', 'request_id': 'req-c957846b-8d45-41a9-b13b-f6314ea40a0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:23.566096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03174c91dbe422a9fe0befb4c79ff2c', 'request_id': 'req-c957846b-8d45-41a9-b13b-f6314ea40a0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:23.577678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] The token cache is being invalidate because project d03174c91dbe422a9fe0befb4c79ff2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:49:23.577967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:23.578590 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c957846b-8d45-41a9-b13b-f6314ea40a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:49:23.580200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1362/2771] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:49:23 2022] DELETE /identity/v3/projects/d03174c91dbe422a9fe0befb4c79ff2c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:49:32.960549 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:33.574748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:36.813304 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:36.815205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:36.900273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b669-3aa5-4de8-893f-24135c0b9729 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:36.971654 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:36.972885 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:36.973169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:36.973442 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:36.975486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:36.988661 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:36.990266 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:37.003060 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1410/2772] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:38.558125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:38.558598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:38.558707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:38.576004 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] MFA Rules not processed for user `6e6e201df4224d51b84849e9898a150d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:49:38.577198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:38.632110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1363/2773] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:49:38.936684 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:49:38.938073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:38.992726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8385ee-ee8f-437c-956e-c4026232ad9e service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:49:38.996284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:49:38.996921 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:49:38.997200 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:49:38.997473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:49:38.999198 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:49:39.008655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:49:39.010062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:49:39.018002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1411/2774] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:49:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:49:48.625087 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=25 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:49.014471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:27.468397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:27.469780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:27.528428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8de610-65ea-46c1-9516-b99fac54f473 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:27.532553 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:27.533241 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:27.533528 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:27.533807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:27.535608 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:27.614993 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:27.616437 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:27.635062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1364/2775] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:50:30.994794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:30.996568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:31.070199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711e299-4fb3-418b-bdd9-f9c5e2504ecf service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:31.075682 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:31.076395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:31.076677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:31.076955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:31.078766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:31.091434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:31.093074 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:31.103955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1412/2776] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:50:35.686624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:35.696672 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139038cf-b5b7-4465-8130-3369c44b87cb service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:35.700198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:35.700835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:35.701115 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:35.701390 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:35.703131 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:35.766276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:35.767691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:35.777334 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1365/2777] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:50:38.853045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:38.854615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:38.868384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf896-10da-4cc1-8260-c521e3e597c7 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:38.875273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:38.876079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:38.876620 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:38.876909 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:38.878761 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:38.892077 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:38.893687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:38.911962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1413/2778] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:50:38.918987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:38.920552 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:38.935016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b99c-801e-42c9-a652-3b80c4bf5877 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:38.940482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:38.941399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:38.941692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:38.941986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] PATH_INFO: `/v3/users/58e0d069005b46268544e890ce08ee2b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:38.943170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e0d069005b46268544e890ce08ee2b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:38.945635 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:38.962328 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:38.995003 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e0d069005b46268544e890ce08ee2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:38.995663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e0d069005b46268544e890ce08ee2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.008612 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e0d069005b46268544e890ce08ee2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.015455 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e0d069005b46268544e890ce08ee2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.016112 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-a8a57604-6e11-40b3-be71-01d00a7d4cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e0d069005b46268544e890ce08ee2b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.029326 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1366/2779] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:50:38 2022] DELETE /identity/v3/users/58e0d069005b46268544e890ce08ee2b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:50:39.203855 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.214405 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81aeb8-edfc-4717-9aff-b94d2e6b4e15 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.271512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.272399 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.272680 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.272978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] PATH_INFO: `/v3/projects/daff3a2fbb1745519fa8ea28c2c084c3` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.274166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] RBAC: Authorizing `identity:delete_project(project_id=daff3a2fbb1745519fa8ea28c2c084c3)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.276066 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.321472 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daff3a2fbb1745519fa8ea28c2c084c3', 'request_id': 'req-0196d0ee-663f-4def-b11d-bfab04d6157e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.327089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daff3a2fbb1745519fa8ea28c2c084c3', 'request_id': 'req-0196d0ee-663f-4def-b11d-bfab04d6157e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.333860 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daff3a2fbb1745519fa8ea28c2c084c3', 'request_id': 'req-0196d0ee-663f-4def-b11d-bfab04d6157e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.344962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] The token cache is being invalidate because project daff3a2fbb1745519fa8ea28c2c084c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:50:39.345263 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.345886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-0196d0ee-663f-4def-b11d-bfab04d6157e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:50:39.347438 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1414/2780] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:50:39 2022] DELETE /identity/v3/projects/daff3a2fbb1745519fa8ea28c2c084c3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:50:39.352372 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.352660 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.352937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.371873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:50:39.373156 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:39.439355 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1367/2781] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:50:39.447527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.447840 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.448123 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.460417 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:39.467237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:50:39.469148 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:39.478299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1415/2782] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:50:39.482738 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.492781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fbf2-3ec2-4156-8249-635f031a58b9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.497289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.497938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.498227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.498502 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.499227 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:50:39.499534 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.501168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.513683 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1368/2783] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:50:39.517970 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.527677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d40a-8310-4cd6-a091-3539a2f47e58 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.532312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.532955 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.533239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.533516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.534205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.536164 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.560577 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1416/2784] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:50:39 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:50:39.564549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.574947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e8eeb-d57b-4045-b3f9-3f5a96a839bf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.632874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.633573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.633851 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.634119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.634891 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.636849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.657461 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1369/2785] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:50:39 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:50:39.660498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.673980 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e633ca-bf1a-4c02-ac47-23995b1c1a37 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.678548 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.679231 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.679516 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.679825 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.680553 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.682147 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.692348 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1417/2786] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:50:39.696874 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.706848 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01a666-03e4-4a25-a4ca-910f5ce8ebb4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.711947 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.712581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.712862 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.713136 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.713853 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.715419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37645f3-ffec-490f-beed-b4dbb659a965 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.717423 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:39.728096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1370/2787] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:50:39.731546 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.741434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8ad82-76c8-422d-ad23-c55a024df9a5 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.746176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.747043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.747320 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.747601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] PATH_INFO: `/v3/projects/bd69e955dd7c42d39026292bad8ea136/users/bc89660f36114834bc57271ef36c66c5/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.762987 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd69e955dd7c42d39026292bad8ea136, user_id=bc89660f36114834bc57271ef36c66c5, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.765685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.779259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1418/2788] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:50:39 2022] PUT /identity/v3/projects/bd69e955dd7c42d39026292bad8ea136/users/bc89660f36114834bc57271ef36c66c5/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:50:39.784549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f711b-e242-47c2-8663-84c682882239 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.784842 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f711b-e242-47c2-8663-84c682882239 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.785119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f711b-e242-47c2-8663-84c682882239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.801752 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-108f711b-e242-47c2-8663-84c682882239 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:50:39.802945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-108f711b-e242-47c2-8663-84c682882239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:39.866322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1371/2789] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:50:39.876749 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:39.878509 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:39.935529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd40f7-264d-4b00-b15a-352ead46578a admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:39.939259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:39.940056 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:39.940346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:39.940618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:39.951412 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:39.952879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:39.962211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1419/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:50:43.686364 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0524b-058e-45e6-bfb5-8a68885df0c6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:43.686896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0524b-058e-45e6-bfb5-8a68885df0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:43.687007 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0524b-058e-45e6-bfb5-8a68885df0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:43.714841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4fb0524b-058e-45e6-bfb5-8a68885df0c6 None None] MFA Rules not processed for user `bc89660f36114834bc57271ef36c66c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:50:43.716178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4fb0524b-058e-45e6-bfb5-8a68885df0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:43.782563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1372/2791] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:50:43.796865 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:43.798339 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:43.876302 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402204a2-ea9c-4099-89fe-2cdf05491c28 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:43.883496 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:43.884293 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:43.884655 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:43.884941 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:43.899519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:43.901319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:43.910568 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1420/2792] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:50:45.230580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7d9c-753a-42be-8a61-1770dc144633 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:45.231117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7d9c-753a-42be-8a61-1770dc144633 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:45.231117 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7d9c-753a-42be-8a61-1770dc144633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:45.249308 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3eda7d9c-753a-42be-8a61-1770dc144633 None None] MFA Rules not processed for user `bc89660f36114834bc57271ef36c66c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:50:45.250443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3eda7d9c-753a-42be-8a61-1770dc144633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:45.258403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1373/2793] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:50:53.906466 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:55.243841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=63 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:52:00.584949 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:00.586331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:00.642457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e3fd0-1cf0-4271-a7fd-8f668671f865 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:00.646236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:00.646879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:00.647176 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:00.647448 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:00.649197 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:00.705355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:00.706690 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:00.715649 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1421/2794] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:10.711463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=48 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:52:14.808730 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:14.809247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:14.809367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:14.826339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] MFA Rules not processed for user `6e6e201df4224d51b84849e9898a150d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:52:14.827556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:14.834445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1374/2795] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:52:14.874223 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:14.875706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:14.886908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483d203-64c2-4203-a683-02a8aa41b868 service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:14.890930 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:14.891560 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:14.891877 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:14.892150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:14.893942 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:14.907833 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:14.909258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:14.920098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1422/2796] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:15.206380 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:15.215919 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0884-d6ae-4126-bf74-272302d3e7fb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:15.220075 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:15.220713 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:15.220998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:15.221276 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:15.223023 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:15.232419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:15.233837 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:15.241787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1375/2797] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:15.713507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:15.722849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682382c-edec-4e7b-8cc5-a5a628b18b9e service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:15.726617 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:15.727347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:15.727703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:15.728115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:15.729996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:15.744357 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:15.745796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac954d-4e36-40db-afdc-75ec30217406 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:15.758084 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1423/2798] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:25.238205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:52:25.753067 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=12 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:52:35.835469 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:35.844149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94859d1a-6f8a-473d-a805-1569881459d2 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:35.847667 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:35.848333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:35.848617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:35.848895 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:35.850626 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:35.912497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:35.913847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:35.923190 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1376/2799] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:45.918931 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=30 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:52:48.574467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:48.574936 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:48.575040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:48.592582 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] MFA Rules not processed for user `bc89660f36114834bc57271ef36c66c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:52:48.593766 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:48.601183 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1424/2800] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:52:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:52:50.900072 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:52:50.901621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:50.912976 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03c17c-103e-42fe-97db-9aa9cbe91642 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:52:50.916658 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:52:50.917306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:52:50.917587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:52:50.917879 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:52:50.919750 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:52:50.930197 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:52:50.931572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba1faf-890d-41b3-89d7-abec219f38b9 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:52:50.940086 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1377/2801] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:52:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:52:58.587908 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:53:00.936167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:54:40.862837 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:40.864935 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:40.983440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e3b34-b81b-4570-86d0-0505076af2c4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:40.989287 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:40.989962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:40.990217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:40.990488 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:40.992298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:41.005711 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:41.007281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:41.019091 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1425/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:54:47.853148 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:47.853680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:47.853680 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:47.871133 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] MFA Rules not processed for user `bc89660f36114834bc57271ef36c66c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:54:47.872868 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:47.880350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1378/2803] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:54:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:54:48.488687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:48.490211 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:48.564311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71fa4-09a8-472c-b3df-70909f798dd5 service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:48.572280 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:48.573000 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:48.573473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:48.573783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:48.576114 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:48.587926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:48.589492 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:48.600717 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1426/2804] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:54:51.749342 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:51.769376 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bd312-df2e-453b-adf6-ad4efc253b90 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:51.793975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:51.795084 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:51.795564 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:51.796066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:51.798110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:51.812295 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:51.814445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:51.831217 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1379/2805] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:54:54.975133 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:54.984842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01702-2de9-4b16-9b47-ad21db4905c6 service placement] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:54.988469 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:54.989105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:54.989386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:54.989665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:54.991390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:55.003888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:55.005346 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:55.015594 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1427/2806] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:54:58.041401 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:58.043980 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:58.058383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c04d8-8229-4eae-a88d-c2f87d6e58ba service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:58.063375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:58.064052 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:58.064333 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:58.064798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:58.066859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:58.079649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:58.081338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:58.092929 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1380/2807] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:54:59.072505 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:54:59.088384 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb481f64-1080-4dfd-bead-1ea934e2ab83 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:54:59.094948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:54:59.095898 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:54:59.096537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:54:59.096880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:54:59.098778 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:54:59.112045 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:54:59.113978 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:54:59.126370 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1428/2808] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:54:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:55:01.513172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:55:01.516174 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:55:01.546416 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841d21-e22b-4bb2-a6cf-33a4235c8b57 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:55:01.556010 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:55:01.557180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:55:01.557542 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:55:01.557825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] PATH_INFO: `/v3/users/6e6e201df4224d51b84849e9898a150d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:55:01.559160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6e201df4224d51b84849e9898a150d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:55:01.561123 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:55:01.630701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6e201df4224d51b84849e9898a150d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:01.631754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6e201df4224d51b84849e9898a150d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:01.645054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6e201df4224d51b84849e9898a150d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:01.656495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6e201df4224d51b84849e9898a150d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:01.657168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6e201df4224d51b84849e9898a150d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:01.671736 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1381/2809] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:55:01 2022] DELETE /identity/v3/users/6e6e201df4224d51b84849e9898a150d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:55:01.870418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:55:01.882956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd46186-f661-4aa3-982d-9f8a7fef33ec service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:55:01.939968 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=100 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:55:01.970550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:55:01.971413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:55:01.971696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:55:01.972117 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] PATH_INFO: `/v3/projects/1bc620581ba8491e90acdb632452d971` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:55:01.973299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc620581ba8491e90acdb632452d971)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:55:01.975191 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:55:02.031601 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc620581ba8491e90acdb632452d971', 'request_id': 'req-ec7a0356-26ec-46ed-bb04-89fbb6931047'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.038023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc620581ba8491e90acdb632452d971', 'request_id': 'req-ec7a0356-26ec-46ed-bb04-89fbb6931047'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.045217 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc620581ba8491e90acdb632452d971', 'request_id': 'req-ec7a0356-26ec-46ed-bb04-89fbb6931047'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.056715 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] The token cache is being invalidate because project 1bc620581ba8491e90acdb632452d971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:55:02.056934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.057571 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.059258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1429/2810] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:55:01 2022] DELETE /identity/v3/projects/1bc620581ba8491e90acdb632452d971 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:55:02.063236 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:55:02.064716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:55:02.078581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02da60-2856-4af9-a5d0-98a481f904be admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:55:02.136846 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:55:02.137723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:55:02.138006 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:55:02.138278 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] PATH_INFO: `/v3/users/ffde35756ff44285bcffa846568af106` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:55:02.139463 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffde35756ff44285bcffa846568af106)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:55:02.141562 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:55:02.185758 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:55:02.192627 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffde35756ff44285bcffa846568af106'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.193382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffde35756ff44285bcffa846568af106'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.207130 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffde35756ff44285bcffa846568af106'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.213986 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffde35756ff44285bcffa846568af106'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.214631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffde35756ff44285bcffa846568af106'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.227803 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1382/2811] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:55:02 2022] DELETE /identity/v3/users/ffde35756ff44285bcffa846568af106 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:55:02.407794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:55:02.409342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:55:02.433790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0356-26ec-46ed-bb04-89fbb6931047 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:55:02.491783 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:55:02.492938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:55:02.493264 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:55:02.493647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] PATH_INFO: `/v3/projects/ff94e39429e84d9381a0b5ad0e15a725` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:55:02.495073 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff94e39429e84d9381a0b5ad0e15a725)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:55:02.497681 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:55:02.547725 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff94e39429e84d9381a0b5ad0e15a725', 'request_id': 'req-a4fa7191-70b6-4255-baa6-66082279e60d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.553269 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff94e39429e84d9381a0b5ad0e15a725', 'request_id': 'req-a4fa7191-70b6-4255-baa6-66082279e60d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.561675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff94e39429e84d9381a0b5ad0e15a725', 'request_id': 'req-a4fa7191-70b6-4255-baa6-66082279e60d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.567181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:55:02.579377 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] The token cache is being invalidate because project ff94e39429e84d9381a0b5ad0e15a725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:55:02.579716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.580555 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:55:02.582946 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1430/2812] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:55:02 2022] DELETE /identity/v3/projects/ff94e39429e84d9381a0b5ad0e15a725 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:55:12.224198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:55:12.576951 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=2 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:55:46.193766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:55:46.195454 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:55:46.286014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d1bea-1662-4ffa-b903-2620264ab89c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:55:46.347100 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:55:46.347751 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:55:46.348062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:55:46.348338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:55:46.350103 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:55:46.361207 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:55:46.362779 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b6bf-b8e6-46c4-a06e-814a90107810 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:55:46.372674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1383/2813] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:55:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:55:56.368447 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=67 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:56:18.310507 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:56:18.312565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:56:18.383652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa7191-70b6-4255-baa6-66082279e60d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:56:18.389104 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e441b994217b4b81be762246ef726545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:56:18.389739 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:56:18.390019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:56:18.390291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:56:18.392029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:56:18.449640 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:56:18.451799 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:56:18.460686 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1431/2814] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:56:28.456562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=56 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:56:35.609962 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:56:35.610475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:56:35.610598 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:56:35.627780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] MFA Rules not processed for user `bc89660f36114834bc57271ef36c66c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:56:35.629221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:56:35.636066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1384/2815] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:56:35.932261 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:56:35.933758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:56:36.008768 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abcfc7-2342-4859-b54e-2b198ac17ace service nova] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:56:36.012323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:56:36.012965 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:56:36.013249 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:56:36.013527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:56:36.015281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:56:36.024638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:56:36.026062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:56:36.034526 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1432/2816] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:56:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:56:45.623448 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:56:46.031059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=35 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:57:51.857402 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:57:51.858901 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:57:51.870792 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb098e6-682c-4bf5-987f-cabe8999fe05 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:57:51.875125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:57:51.875789 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:57:51.876110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:57:51.876389 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:57:51.878119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:57:51.887871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:57:51.889420 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:57:51.897945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1385/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:57:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:01.894360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:02.552222 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:02.554563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:02.634081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e561f22-c41a-4acf-971e-1f2f0fe015cc service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:02.637791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:02.638431 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:02.638719 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:02.638996 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:02.641262 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:02.654232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:02.655638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:02.666985 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1433/2818] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:10.671315 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:10.672949 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:10.684567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c2fa-5ab4-4903-886f-8e64271dd1a8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:10.687990 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:10.688619 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:10.688899 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:10.689170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:10.690867 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:10.704391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:10.705795 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:10.716699 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1386/2819] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:12.662225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=35 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:20.711716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:21.066156 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:21.075567 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c61883-90f6-4895-b1a2-dd40bf166a0f service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:21.079938 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:21.080596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:21.080876 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:21.081149 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:21.082944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:21.164827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:21.166239 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:21.185181 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1434/2820] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:24.007205 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.008636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:24.024264 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a048d-133c-4fdf-a19f-a7394d54c81a service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.029510 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.030172 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.030459 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.030737 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.032832 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:24.048810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.050723 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.057344 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.058834 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:24.061682 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1387/2821] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:24.071677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695afaa0-9fad-415b-a540-237dcc1ab912 service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.076643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.077593 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.077858 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.078595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] PATH_INFO: `/v3/users/bc89660f36114834bc57271ef36c66c5` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.080189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc89660f36114834bc57271ef36c66c5)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.082140 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.134362 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc89660f36114834bc57271ef36c66c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.147618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc89660f36114834bc57271ef36c66c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.148395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc89660f36114834bc57271ef36c66c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.149019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc89660f36114834bc57271ef36c66c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.156281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-af8e44df-4d2e-4cc4-a713-2383efb58c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc89660f36114834bc57271ef36c66c5'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.170413 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1435/2822] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:24 2022] DELETE /identity/v3/users/bc89660f36114834bc57271ef36c66c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:24.348701 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.366110 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5866833-812a-4597-8e57-c09024dbccd1 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.428898 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.429780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.430096 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.430374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] PATH_INFO: `/v3/projects/bd69e955dd7c42d39026292bad8ea136` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.431570 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd69e955dd7c42d39026292bad8ea136)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.433493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.479856 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd69e955dd7c42d39026292bad8ea136', 'request_id': 'req-95317827-4c1e-443a-b60f-5f0179e97ea6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.485983 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd69e955dd7c42d39026292bad8ea136', 'request_id': 'req-95317827-4c1e-443a-b60f-5f0179e97ea6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.492014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd69e955dd7c42d39026292bad8ea136', 'request_id': 'req-95317827-4c1e-443a-b60f-5f0179e97ea6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.504366 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] The token cache is being invalidate because project bd69e955dd7c42d39026292bad8ea136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:24.504718 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.505593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95317827-4c1e-443a-b60f-5f0179e97ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:24.507132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1388/2823] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:24 2022] DELETE /identity/v3/projects/bd69e955dd7c42d39026292bad8ea136 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:24.512246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.512537 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.512815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.566343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:24.569899 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:24.571529 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:24.629658 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1436/2824] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:24.637617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.637904 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.638178 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.656019 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:24.657697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:24.665872 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1389/2825] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:24.669891 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.679960 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea669b-5700-4d4d-9fa4-a26383f626af None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.684627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.685310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.685587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.685862 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.686578 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:58:24.686879 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.688494 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.699918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1437/2826] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:58:24.703979 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.713158 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642a99-64dd-4cf3-87c3-58a79c899f60 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.717781 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.718421 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.718704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.718982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.720209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.722106 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.756849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1390/2827] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:24 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:58:24.760895 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.770716 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd59add-3c1a-4e22-bdd1-5a44a9ce9d97 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.825572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.826258 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.826539 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.826814 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.827512 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.829963 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.849049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1438/2828] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:24 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:24.853053 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.863356 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc7092-1d29-4d6d-b911-7801b58d45ad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.868214 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.868873 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.869157 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.869434 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.870160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.871796 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.882142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1391/2829] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:24.885944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.896218 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784215a1-c810-4d74-8dee-66f63fdd622b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.900777 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.901423 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.901704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.901979 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.902700 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.904325 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202103ca-69d6-4b9f-87be-f2f6f75973c3 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.914734 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1439/2830] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:24.919228 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:24.929049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59971ac-792e-4440-875b-37c96b9ab147 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:24.933595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:24.934456 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.934735 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.935008 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] PATH_INFO: `/v3/projects/fc67e5475dc241ee8b5bfbbb75fee4c6/users/b9c2f74a019e4315bfcf0e1d83162c29/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:24.949250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc67e5475dc241ee8b5bfbbb75fee4c6, user_id=b9c2f74a019e4315bfcf0e1d83162c29, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:24.951890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:24.964870 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1392/2831] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:24 2022] PUT /identity/v3/projects/fc67e5475dc241ee8b5bfbbb75fee4c6/users/b9c2f74a019e4315bfcf0e1d83162c29/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:24.976195 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:24.976484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:24.976789 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.002251 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] MFA Rules not processed for user `b9c2f74a019e4315bfcf0e1d83162c29`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:25.003444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:25.065155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1440/2832] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:58:25.069524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.075177 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:25.132020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856e0-9ff2-4f9d-941e-6762c052c3eb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.136732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.137330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.137614 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.137890 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.138596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.140519 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.166391 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1393/2833] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:25 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:25.170476 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.175967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:25.183558 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55f2f-6ae9-4029-9808-e5e846192d7a None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.238730 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.239386 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.239664 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.239962 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.240645 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.242527 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.262256 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1441/2834] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:25 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:25.266772 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.276934 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23791e-aea4-43a3-880f-b6774bc33a6f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.281521 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.282173 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.282453 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.282726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.283450 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.285070 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.295377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1394/2835] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:25.299350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.309299 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81d3d-0f95-4b42-9c30-f2a397433bfa admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.314449 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.315315 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.315596 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.315902 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] PATH_INFO: `/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e/users/17379ab7e401436eba609bc651d4e76e/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.331061 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4609a64a9c624aff9e974ffbc4e7c12e, user_id=17379ab7e401436eba609bc651d4e76e, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.333741 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.346309 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1442/2836] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:25 2022] PUT /identity/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e/users/17379ab7e401436eba609bc651d4e76e/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:25.350091 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.359998 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56899ab8-a375-4cc1-9d3b-58c6c7c3e3e6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.416226 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.416816 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.417101 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.417381 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.418105 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.419704 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e3245-6da1-4a59-b3d1-896f6d656fff admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.430462 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1395/2837] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:25.434587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.445631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93128c8-3884-49a8-aaa3-f88ffb87a565 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.450160 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.451019 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.451298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.451569 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] PATH_INFO: `/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e/users/17379ab7e401436eba609bc651d4e76e/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.453633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4609a64a9c624aff9e974ffbc4e7c12e, user_id=17379ab7e401436eba609bc651d4e76e, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.456795 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.466613 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1443/2838] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:25 2022] PUT /identity/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e/users/17379ab7e401436eba609bc651d4e76e/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:25.476591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.476886 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.477168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.504743 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] MFA Rules not processed for user `17379ab7e401436eba609bc651d4e76e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:25.506271 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:25.569497 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:25.573871 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1396/2839] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:25.585150 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.587002 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:25.644427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0369-3613-427d-a7e4-56c3de604332 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.648006 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.648636 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.648912 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.649180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.657923 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.659330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.668602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1444/2840] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:25.716688 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:25.718097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:25.729991 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663b064-3d37-47c5-a1fd-b3cfbedbeea3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:25.733625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:25.734262 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:25.734550 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:25.735387 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:25.795279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:25.796755 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a67539-14f7-4fa2-a72e-e7df3f914233 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:25.811914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1397/2841] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:29.030265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:29.086525 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:29.090787 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f593eb-3471-431f-8e21-c717776ecaba service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:29.096647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:29.097523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:29.097806 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:29.098081 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] PATH_INFO: `/v3/users/b9c2f74a019e4315bfcf0e1d83162c29` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:29.099307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c2f74a019e4315bfcf0e1d83162c29)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:29.101255 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:29.147605 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c2f74a019e4315bfcf0e1d83162c29'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.159948 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c2f74a019e4315bfcf0e1d83162c29'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.160616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c2f74a019e4315bfcf0e1d83162c29'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.161225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c2f74a019e4315bfcf0e1d83162c29'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.168021 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c2f74a019e4315bfcf0e1d83162c29'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.181265 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1445/2842] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:29 2022] DELETE /identity/v3/users/b9c2f74a019e4315bfcf0e1d83162c29 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:29.185847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:29.186344 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:29.186466 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:29.203168 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:29.204399 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:29.261982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1398/2843] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:29.271484 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:29.272928 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:29.330330 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1ec02-19b1-4ed9-b338-7951631dd4fd admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:29.333929 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:29.334564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:29.334844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:29.335119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:29.343977 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:29.345933 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:29.355234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1446/2844] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:29.626578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:29.628126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:29.643668 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8e00c-11d0-427b-9d1f-939fa87b120d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:29.649330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:29.650195 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:29.650476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:29.650762 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] PATH_INFO: `/v3/projects/fc67e5475dc241ee8b5bfbbb75fee4c6` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:29.652000 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc67e5475dc241ee8b5bfbbb75fee4c6)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:29.653896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:29.701418 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc67e5475dc241ee8b5bfbbb75fee4c6', 'request_id': 'req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.707482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc67e5475dc241ee8b5bfbbb75fee4c6', 'request_id': 'req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.713523 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc67e5475dc241ee8b5bfbbb75fee4c6', 'request_id': 'req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.726024 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] The token cache is being invalidate because project fc67e5475dc241ee8b5bfbbb75fee4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:29.726289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.726908 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.728495 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1399/2845] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:29 2022] DELETE /identity/v3/projects/fc67e5475dc241ee8b5bfbbb75fee4c6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:29.732434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:29.741842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc616-e251-4110-abf2-f91b960213ef service neutron] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:29.798556 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:29.799419 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:29.799696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:29.799999 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] PATH_INFO: `/v3/users/17379ab7e401436eba609bc651d4e76e` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:29.801173 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17379ab7e401436eba609bc651d4e76e)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:29.803118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:29.825607 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:29.852312 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17379ab7e401436eba609bc651d4e76e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.864954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17379ab7e401436eba609bc651d4e76e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.865677 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17379ab7e401436eba609bc651d4e76e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.866311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17379ab7e401436eba609bc651d4e76e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.874934 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-14915533-99b6-4432-8d31-b83699b19382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17379ab7e401436eba609bc651d4e76e'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:29.890564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1447/2846] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:29 2022] DELETE /identity/v3/users/17379ab7e401436eba609bc651d4e76e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:30.156363 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.157836 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:30.171263 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22ee66-b3a2-4f2a-9447-2cc66bd84ce8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.197814 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:30.237457 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.238245 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.238530 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.238805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] PATH_INFO: `/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.240020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4609a64a9c624aff9e974ffbc4e7c12e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.241921 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.287994 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4609a64a9c624aff9e974ffbc4e7c12e', 'request_id': 'req-86b0a121-1234-4e2e-a470-29dede6750d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:30.293409 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4609a64a9c624aff9e974ffbc4e7c12e', 'request_id': 'req-86b0a121-1234-4e2e-a470-29dede6750d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:30.299952 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4609a64a9c624aff9e974ffbc4e7c12e', 'request_id': 'req-86b0a121-1234-4e2e-a470-29dede6750d0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:30.311698 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] The token cache is being invalidate because project 4609a64a9c624aff9e974ffbc4e7c12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:30.312014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:30.312624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-86b0a121-1234-4e2e-a470-29dede6750d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:30.314256 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1400/2847] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:30 2022] DELETE /identity/v3/projects/4609a64a9c624aff9e974ffbc4e7c12e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:30.319874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.320532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.320532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.338705 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:30.339918 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:30.398317 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1448/2848] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:58:30.407578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.408397 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.408697 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.427419 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:30.428560 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:30.436572 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1401/2849] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:30.441434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.451252 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7baa0d-86ff-43c0-863a-9ccfba8701f4 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.456538 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.457189 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.457473 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.457748 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.458503 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:58:30.458809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.460455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.471940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1449/2850] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:58:30.476129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.485350 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd6090-711f-449a-85e3-748a9771238b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.490078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.490663 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.490945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.491218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.491940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.493828 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.519844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1402/2851] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:30 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:30.572712 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.583471 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be98cf-3c4b-4519-9e02-7302cfcbb349 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.639052 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.639710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.640018 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.640291 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.640974 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.643451 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.662427 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1450/2852] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:30 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:30.666556 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.676807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9195-75bd-4984-9929-5491a3d38080 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.681422 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.682238 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.682522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.682798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.683555 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.685247 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.696147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1403/2853] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:30.700050 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.710323 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39603-dfa6-43de-a14c-1d27ebfb9ddb admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.714943 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.715581 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.715894 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.716169 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.716880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.718440 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f12b4-0344-424a-9fe2-d3e0d97acda8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.728080 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1451/2854] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:30.731930 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.741649 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbafdf5-ea5c-4d8b-bc3c-91234435ac73 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.746175 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.747021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.747301 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.747576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] PATH_INFO: `/v3/projects/b8030dc289614112bd594339526eece8/users/940495780e294ac0b0660c910cbdbc55/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.762351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8030dc289614112bd594339526eece8, user_id=940495780e294ac0b0660c910cbdbc55, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.765002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.778200 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1404/2855] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:30 2022] PUT /identity/v3/projects/b8030dc289614112bd594339526eece8/users/940495780e294ac0b0660c910cbdbc55/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:30.787984 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.788270 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.788550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.795247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:30.816102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] MFA Rules not processed for user `940495780e294ac0b0660c910cbdbc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:30.817295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:30.879728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1452/2856] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:30.891255 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:30.892985 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:30.945905 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:30.950490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754acf2-fa23-4b5c-af8c-d3314daad00d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:30.954975 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:30.955732 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:30.956076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:30.956529 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:30.964707 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:30.966118 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebade9c-4f36-457c-9402-2ca102b69b33 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:30.974766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1405/2857] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:30.993852 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.055162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d0153-21a8-4e70-81da-d612fa4a8ef9 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.059685 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.060565 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.060846 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.061119 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] PATH_INFO: `/v3/users/940495780e294ac0b0660c910cbdbc55` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.062297 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=940495780e294ac0b0660c910cbdbc55)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.064809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.109647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940495780e294ac0b0660c910cbdbc55'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.122455 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940495780e294ac0b0660c910cbdbc55'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.123122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940495780e294ac0b0660c910cbdbc55'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.123738 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940495780e294ac0b0660c910cbdbc55'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.131564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940495780e294ac0b0660c910cbdbc55'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.144543 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1453/2858] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:30 2022] DELETE /identity/v3/users/940495780e294ac0b0660c910cbdbc55 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:31.149005 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.149294 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.149567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.167876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:31.169066 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:31.227011 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1406/2859] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:31.237234 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.238598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:31.252633 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:31.295791 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff76099-2ac8-4662-8c2b-a062e4c9de38 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.299420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.300128 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.300631 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.301029 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.310627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.312089 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5071-a8f3-4a07-9db4-b335c94802fd service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.321813 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1454/2860] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:31.582041 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.584398 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:31.597624 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cec99c-d807-4063-9607-a7f9a41fd8d9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.603180 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.604595 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.604807 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.605102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] PATH_INFO: `/v3/projects/b8030dc289614112bd594339526eece8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.606474 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8030dc289614112bd594339526eece8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.608377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.656014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8030dc289614112bd594339526eece8', 'request_id': 'req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.661654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8030dc289614112bd594339526eece8', 'request_id': 'req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.668015 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8030dc289614112bd594339526eece8', 'request_id': 'req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.680320 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] The token cache is being invalidate because project b8030dc289614112bd594339526eece8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:31.680604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.681221 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-71ab43c8-21d9-44e2-98aa-17a1f4a57c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:31.682725 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1407/2861] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:31 2022] DELETE /identity/v3/projects/b8030dc289614112bd594339526eece8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:31.687780 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.688079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.688355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.706100 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:31.707298 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:31.766328 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1455/2862] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:58:31.773778 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.774063 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.774338 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.792059 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:31.793244 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:31.801591 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1408/2863] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:31.805758 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.815961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98c7d-22dd-448e-8cb2-1008b13676ec None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.820638 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.821281 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.821566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.821843 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.822572 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:58:31.822882 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.824506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.839203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1456/2864] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:58:31.844076 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.855020 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e76bda-c213-4d0a-8105-e71013d087a6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.860482 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.861637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.861967 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.862273 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.866887 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.869747 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.899309 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1409/2865] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:31 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:31.904283 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:31.914665 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d989aa-f735-414e-b7b1-e80590500d2b admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:31.968652 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:31.969308 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:31.969595 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:31.969867 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:31.971054 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:31.972966 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:31.993467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1457/2866] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:31 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:58:31.997567 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.008119 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be02f7-c5e4-4a88-b650-c939383776ed admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.012719 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.013375 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.013661 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.013937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.014659 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.016303 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.026628 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1410/2867] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:32.030437 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.041023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e4795-5003-46cd-84fc-946184b5f135 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.045647 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.046324 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.046611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.046886 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.047611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.049841 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36058c-f413-4aea-8261-ff5c829191e8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.059616 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1458/2868] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:32.063288 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.073139 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101eccb-ffde-4b5a-a8e5-84e794f52aad admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.077703 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.078568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.078849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.079127 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] PATH_INFO: `/v3/projects/4a495387a97543219f6ecf9f7e9ec065/users/85242e9b7a6748f6a0219063e548338f/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.093414 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a495387a97543219f6ecf9f7e9ec065, user_id=85242e9b7a6748f6a0219063e548338f, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.096580 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.109374 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1411/2869] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:32 2022] PUT /identity/v3/projects/4a495387a97543219f6ecf9f7e9ec065/users/85242e9b7a6748f6a0219063e548338f/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:32.121207 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.121517 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.121794 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.149335 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] MFA Rules not processed for user `85242e9b7a6748f6a0219063e548338f`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:32.150510 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:32.159859 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:32.216310 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1459/2870] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:32.228274 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.230054 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:32.244716 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:32.289805 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d408-7e2d-4138-a8b8-16e3b98482da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.293330 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.293973 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.294252 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.294525 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.304279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.305726 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe3d80-cca7-4bd4-84e9-ef78da310fbd service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.314827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1412/2871] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:32.334305 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.395246 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ecac-0a03-4b1a-9dab-31717b97dd1e None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.399821 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.400675 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.400954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.401240 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] PATH_INFO: `/v3/users/85242e9b7a6748f6a0219063e548338f` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.402404 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85242e9b7a6748f6a0219063e548338f)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.404364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.449838 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85242e9b7a6748f6a0219063e548338f'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:32.461989 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85242e9b7a6748f6a0219063e548338f'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:32.462663 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85242e9b7a6748f6a0219063e548338f'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:32.463289 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85242e9b7a6748f6a0219063e548338f'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:32.470726 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85242e9b7a6748f6a0219063e548338f'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:32.483967 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1460/2872] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:32 2022] DELETE /identity/v3/users/85242e9b7a6748f6a0219063e548338f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:32.488311 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.488602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.488877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.505953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:32.507145 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:32.566322 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1413/2873] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:32.575873 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.577284 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:32.612319 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:32.636643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a472-d0fe-4f6a-b105-e51d42a63e9f admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.640248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.640910 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.641193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.641467 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.650815 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.652273 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702625c-fa63-4cec-b927-ce2955f1d36f service neutron] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.662132 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1461/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:32.925518 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:32.927062 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:32.940662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20758b6-4d9f-4f52-a8d7-f2ca318f0815 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:32.945289 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:32.946841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:32.947475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:32.947849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] PATH_INFO: `/v3/projects/4a495387a97543219f6ecf9f7e9ec065` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:32.949002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a495387a97543219f6ecf9f7e9ec065)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:32.950896 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:32.997790 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a495387a97543219f6ecf9f7e9ec065', 'request_id': 'req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:33.003826 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a495387a97543219f6ecf9f7e9ec065', 'request_id': 'req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:33.010146 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a495387a97543219f6ecf9f7e9ec065', 'request_id': 'req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:33.022346 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] The token cache is being invalidate because project 4a495387a97543219f6ecf9f7e9ec065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:33.022633 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:33.023250 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-dd5dd8c8-3f64-4127-b4e5-fe32345ec380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:33.024824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1414/2875] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:32 2022] DELETE /identity/v3/projects/4a495387a97543219f6ecf9f7e9ec065 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:33.029618 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266999f-bd13-4958-aca6-beaed325692b None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.029905 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266999f-bd13-4958-aca6-beaed325692b None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.030177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266999f-bd13-4958-aca6-beaed325692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.047062 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-f266999f-bd13-4958-aca6-beaed325692b None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:33.048757 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f266999f-bd13-4958-aca6-beaed325692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:33.106079 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1462/2876] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:58:33.114258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.114549 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.114827 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.133476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:33.134674 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:33.142578 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1415/2877] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:33.146706 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266999f-bd13-4958-aca6-beaed325692b None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.156295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266999f-bd13-4958-aca6-beaed325692b None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.160722 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.161392 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.161671 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.161940 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] PATH_INFO: `/v3/domains` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.162653 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:58:33.162954 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.164562 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.176701 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1463/2878] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:58:33.180654 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.190705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672f844-2587-460d-9c11-3a38c3eeb38f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.195396 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.196078 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.196361 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.196637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.197335 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.199218 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.223855 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1416/2879] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:33.228043 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.238522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddec11-e8ea-42f6-b0ce-9932999e969d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.294129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.294878 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.295166 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.295446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.296208 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.298672 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.319118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1464/2880] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:58:33.323138 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.333581 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff0b4e-38f3-4fd8-a58f-ba6cf9f7832e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.338243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.338902 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.339187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.339465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.340225 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.341810 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.343845 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:33.356777 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1417/2881] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:33.357641 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.368206 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281d22-976d-4cc4-8248-00bd7e5937e8 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.372826 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.373486 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.373771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.374055 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.374785 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.377023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193ddf0-9e59-4357-bab2-0b986bb0abb7 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.387247 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1465/2882] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:58:33.391587 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.401601 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f171c-c657-4f55-926c-754fef29f050 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.406488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.407371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.408198 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.408475 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] PATH_INFO: `/v3/projects/79296c94c20c463eb2965ae138750876/users/ef3a888797854e169776c6dd062e9844/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.422728 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79296c94c20c463eb2965ae138750876, user_id=ef3a888797854e169776c6dd062e9844, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.425941 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.440243 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1418/2883] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:33 2022] PUT /identity/v3/projects/79296c94c20c463eb2965ae138750876/users/ef3a888797854e169776c6dd062e9844/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:33.450376 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589579-25f4-4d76-bad4-1b429495fced None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.450673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589579-25f4-4d76-bad4-1b429495fced None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.450950 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589579-25f4-4d76-bad4-1b429495fced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.477566 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-40589579-25f4-4d76-bad4-1b429495fced None None] MFA Rules not processed for user `ef3a888797854e169776c6dd062e9844`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:33.478832 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-40589579-25f4-4d76-bad4-1b429495fced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:33.511359 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:33.552627 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1466/2884] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:33.557621 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.622149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1a789-c2db-4568-bedb-aedfd3e04b25 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.627575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.628352 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.628636 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.628914 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.629602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.631783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.656593 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1419/2885] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:33.660707 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40589579-25f4-4d76-bad4-1b429495fced None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.671203 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40589579-25f4-4d76-bad4-1b429495fced None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.726585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.727248 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.727531 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.727831 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.728522 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.730420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.749762 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1467/2886] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:33.753858 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.765392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da01a8-ef45-4662-9cb6-49e18c6d444c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.770160 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.770841 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.771125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.771403 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.772166 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.773754 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.784642 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1420/2887] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:33.788710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.799083 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd2fd-dc30-421c-ac20-8ed7ddabd7b9 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.803689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.804361 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.804642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.804917 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.805642 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.807215 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a219d-3605-4ea9-8806-1b694581838f admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.817040 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1468/2888] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:33.820987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:33.831604 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5df436-5608-4164-a48c-f5d2580902fe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:33.836329 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:33.837224 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.837509 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.837791 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] PATH_INFO: `/v3/projects/b0457e0905eb4f52a09d0fd3bac5161d/users/181b32a881fd4a0eab66bd06c0a81c66/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.856746 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0457e0905eb4f52a09d0fd3bac5161d, user_id=181b32a881fd4a0eab66bd06c0a81c66, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:33.859943 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:33.873021 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1421/2889] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:33 2022] PUT /identity/v3/projects/b0457e0905eb4f52a09d0fd3bac5161d/users/181b32a881fd4a0eab66bd06c0a81c66/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:33.884550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:33.884844 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:33.885129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:33.912592 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] MFA Rules not processed for user `181b32a881fd4a0eab66bd06c0a81c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:33.913796 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:33.976742 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1469/2890] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:33.981373 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.042825 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa406-71e7-4663-b56d-b9f640192ccd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.047620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.048314 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.048602 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.052615 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.053953 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.055889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.082473 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1422/2891] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:33 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:34.085506 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.097403 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40a5f5-593c-4921-b22a-04f7962e96f3 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.136786 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:34.161331 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.162049 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.162279 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.162550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.163232 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.165143 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.184462 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1470/2892] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:34 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:34.189592 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.199617 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afadb51-113f-4f32-a7f5-21e83be317bb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.204191 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.205445 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.205721 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.205989 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.206702 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.208292 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.210258 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:34.219838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1423/2893] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:34.223563 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.233398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d99b2a-a67b-460c-86a7-a8e80c01b1d2 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.238180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.239374 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.239657 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.239957 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] PATH_INFO: `/v3/projects/eb6db06d6a0c4d73a6ef764820406119/users/86363537f9be41c499b8e1771b35b895/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.255178 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6db06d6a0c4d73a6ef764820406119, user_id=86363537f9be41c499b8e1771b35b895, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.257809 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.270805 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1471/2894] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:34 2022] PUT /identity/v3/projects/eb6db06d6a0c4d73a6ef764820406119/users/86363537f9be41c499b8e1771b35b895/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:58:34.274576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.284441 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467be6-a8ee-4620-abee-e834bd2821fc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.340631 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.341290 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.341565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.341838 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.342551 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.344170 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dbbcf-5889-4ff9-988d-53acd4dfdf0d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.353783 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1424/2895] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:34.357819 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.367827 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c3ec0-fc96-428b-baa9-38313cf0619d admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.372352 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.373216 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.373498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.373771 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] PATH_INFO: `/v3/projects/eb6db06d6a0c4d73a6ef764820406119/users/86363537f9be41c499b8e1771b35b895/roles/d9ec2045400649bdb6f718caf2f62488` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.375673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6db06d6a0c4d73a6ef764820406119, user_id=86363537f9be41c499b8e1771b35b895, role_id=d9ec2045400649bdb6f718caf2f62488)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.378802 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.388926 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1472/2896] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:34 2022] PUT /identity/v3/projects/eb6db06d6a0c4d73a6ef764820406119/users/86363537f9be41c499b8e1771b35b895/roles/d9ec2045400649bdb6f718caf2f62488 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:58:34.399465 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.399801 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.400074 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.425865 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] MFA Rules not processed for user `86363537f9be41c499b8e1771b35b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:34.427077 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:34.498611 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1425/2897] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:34.510398 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:34.512162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:34.568587 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa380c-5b22-4b56-97cc-424f34b4dace admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:34.572205 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:34.572839 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:34.573120 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:34.573395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:34.628842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:34.630544 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:34.639391 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1473/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:36.102383 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:36.103808 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:36.115823 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea042fe8-5b1c-4bc8-84e8-208ed8e73682 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:36.119428 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:36.120099 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:36.120382 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:36.120657 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:36.176780 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:36.178126 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:36.187044 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1426/2899] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:36.687498 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:36.696769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8d1b-20a2-47db-8bbd-5e33bf644b5b service cinder] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:36.700295 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51875df6e7b1444cb31e793aa8727363', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:36.700919 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:36.701199 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:36.701468 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:36.710959 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:36.712445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee81912-0783-4f80-a74f-33db88dfebb1 service cinder] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:36.721643 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1474/2900] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:40.704586 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:40.723566 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:40.768605 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ac69-c601-4e21-8e75-50e0682aaef8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:40.773147 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:40.774016 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:40.774299 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:40.774576 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] PATH_INFO: `/v3/users/ef3a888797854e169776c6dd062e9844` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:40.775784 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3a888797854e169776c6dd062e9844)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:40.777691 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:40.823415 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3a888797854e169776c6dd062e9844'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:40.824068 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3a888797854e169776c6dd062e9844'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:40.836202 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3a888797854e169776c6dd062e9844'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:40.842938 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3a888797854e169776c6dd062e9844'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:40.843573 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3a888797854e169776c6dd062e9844'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:40.856219 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1427/2901] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:40 2022] DELETE /identity/v3/users/ef3a888797854e169776c6dd062e9844 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:40.860842 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:40.861311 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:40.861435 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:40.879129 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:40.880602 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:40.905944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:40.940728 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1475/2902] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:40.950496 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:40.951889 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:41.009388 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b3b9-d4cf-424d-bc4d-2d63b3ba1a5a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:41.012916 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:41.013558 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:41.013835 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:41.014107 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:41.022912 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:41.024360 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:41.033625 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1428/2903] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:41.559007 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:41.560514 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:41.576193 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a125-f763-42bd-a2f9-47aa5c9068fc None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:41.583687 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:41.584523 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:41.584696 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:41.585172 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] PATH_INFO: `/v3/projects/79296c94c20c463eb2965ae138750876` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:41.586383 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=79296c94c20c463eb2965ae138750876)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:41.588497 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:41.638611 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79296c94c20c463eb2965ae138750876', 'request_id': 'req-75c3d146-3e71-466f-8d54-553dc0e3bc1c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.644141 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79296c94c20c463eb2965ae138750876', 'request_id': 'req-75c3d146-3e71-466f-8d54-553dc0e3bc1c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.651118 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79296c94c20c463eb2965ae138750876', 'request_id': 'req-75c3d146-3e71-466f-8d54-553dc0e3bc1c'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.662550 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] The token cache is being invalidate because project 79296c94c20c463eb2965ae138750876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:41.662836 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.663457 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.664993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1476/2904] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:41 2022] DELETE /identity/v3/projects/79296c94c20c463eb2965ae138750876 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:41.668942 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:41.679367 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f1d13-f24d-4fbb-8bf5-5c321f914a59 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:41.716774 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:41.736092 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:41.737031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:41.737306 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:41.737575 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] PATH_INFO: `/v3/users/181b32a881fd4a0eab66bd06c0a81c66` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:41.738748 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181b32a881fd4a0eab66bd06c0a81c66)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:41.740692 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:41.785705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181b32a881fd4a0eab66bd06c0a81c66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.786287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181b32a881fd4a0eab66bd06c0a81c66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.798009 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181b32a881fd4a0eab66bd06c0a81c66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.804945 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181b32a881fd4a0eab66bd06c0a81c66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.805589 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181b32a881fd4a0eab66bd06c0a81c66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:41.818717 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1429/2905] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:41 2022] DELETE /identity/v3/users/181b32a881fd4a0eab66bd06c0a81c66 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:42.094993 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:42.096624 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:42.110105 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3d146-3e71-466f-8d54-553dc0e3bc1c admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:42.166179 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:42.167023 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.167307 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.167583 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] PATH_INFO: `/v3/projects/b0457e0905eb4f52a09d0fd3bac5161d` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.168782 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0457e0905eb4f52a09d0fd3bac5161d)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:42.170693 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:42.215688 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0457e0905eb4f52a09d0fd3bac5161d', 'request_id': 'req-dcfd3827-10b2-4f53-811e-594cf94a6863'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.220290 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:42.221463 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0457e0905eb4f52a09d0fd3bac5161d', 'request_id': 'req-dcfd3827-10b2-4f53-811e-594cf94a6863'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.229168 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0457e0905eb4f52a09d0fd3bac5161d', 'request_id': 'req-dcfd3827-10b2-4f53-811e-594cf94a6863'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.240102 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] The token cache is being invalidate because project b0457e0905eb4f52a09d0fd3bac5161d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:42.240390 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.241016 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.242519 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1477/2906] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:42 2022] DELETE /identity/v3/projects/b0457e0905eb4f52a09d0fd3bac5161d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:42.246430 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:42.247830 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:42.261339 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4f99-4347-4d25-b03d-64cf869462d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:42.318479 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:42.319279 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.319563 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.319877 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] PATH_INFO: `/v3/users/86363537f9be41c499b8e1771b35b895` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.321058 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86363537f9be41c499b8e1771b35b895)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:42.322969 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:42.365876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86363537f9be41c499b8e1771b35b895'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.366522 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86363537f9be41c499b8e1771b35b895'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.378371 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:42.381129 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86363537f9be41c499b8e1771b35b895'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.389940 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86363537f9be41c499b8e1771b35b895'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.390620 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e5460455-b5c8-474c-b0c6-2674c9ab628a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86363537f9be41c499b8e1771b35b895'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.403885 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1430/2907] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:58:42 2022] DELETE /identity/v3/users/86363537f9be41c499b8e1771b35b895 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:42.672692 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:42.674177 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:42.687342 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3827-10b2-4f53-811e-594cf94a6863 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:42.744790 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:42.745956 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.746259 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.746533 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] PATH_INFO: `/v3/projects/eb6db06d6a0c4d73a6ef764820406119` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.747710 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6db06d6a0c4d73a6ef764820406119)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:42.749635 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:42.795615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6db06d6a0c4d73a6ef764820406119', 'request_id': 'req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.801343 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6db06d6a0c4d73a6ef764820406119', 'request_id': 'req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.808424 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6db06d6a0c4d73a6ef764820406119', 'request_id': 'req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.820070 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] The token cache is being invalidate because project eb6db06d6a0c4d73a6ef764820406119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:58:42.820355 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.820994 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-fa09fc8f-365b-400e-8b17-bbb2f776b0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:58:42.822615 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1478/2908] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:58:42 2022] DELETE /identity/v3/projects/eb6db06d6a0c4d73a6ef764820406119 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:42.827476 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.827982 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.828134 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.846002 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:42.847187 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:42.905637 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1431/2909] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:42.913444 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.913756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.914036 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.932236 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:42.933420 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:42.941689 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1479/2910] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:42.946965 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:42.957297 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8048d-d9c9-4b9c-85bb-fadb1c26ef12 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:42.961913 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:42.962565 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.962847 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.963125 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.963894 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:58:42.964206 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:42.965809 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:42.977545 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1432/2911] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jul 20 15:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:58:42.981676 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:42.990874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a44f-8bac-4fec-aa0a-4bb29355ed23 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:42.995446 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:42.996115 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:42.996395 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:42.996669 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] PATH_INFO: `/v3/projects` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:42.997350 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:42.999221 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.025464 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1480/2912] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jul 20 15:58:42 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:43.029444 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:43.039443 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e7c52-284f-45d6-bac0-4a1bbb5cc014 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:43.094849 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:43.095490 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.095787 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.096149 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.096788 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:43.098678 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.118028 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1433/2913] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jul 20 15:58:43 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:58:43.121945 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:43.132857 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cdfe9-903b-46c3-a0aa-d4b6d5d90358 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:43.137426 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:43.138072 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.138418 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.138729 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.139453 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:43.141162 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.151880 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1481/2914] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:43.155559 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:43.165493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49672868-774c-403a-aabb-9be463705aa0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:43.170014 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:43.170656 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.170937 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.171209 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.171957 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:43.173516 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62d707-ac11-4b9f-92f7-04b1e96364a3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.183933 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1434/2915] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jul 20 15:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:58:43.187626 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:43.197847 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145a70-335c-4d03-bff7-92b717269963 admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:43.202347 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:43.203201 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.203482 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.203779 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] PATH_INFO: `/v3/projects/ed624cca84614be5addb3b8eb7eaf288/users/f57c7a722fb54142a90a368952b1368a/roles/b87cb4e1db1f4f789877ca5e28243e31` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.219005 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed624cca84614be5addb3b8eb7eaf288, user_id=f57c7a722fb54142a90a368952b1368a, role_id=b87cb4e1db1f4f789877ca5e28243e31)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:43.221662 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.233674 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1482/2916] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jul 20 15:58:43 2022] PUT /identity/v3/projects/ed624cca84614be5addb3b8eb7eaf288/users/f57c7a722fb54142a90a368952b1368a/roles/b87cb4e1db1f4f789877ca5e28243e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:58:43.244234 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.244705 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.245081 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.271844 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] MFA Rules not processed for user `f57c7a722fb54142a90a368952b1368a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:58:43.273049 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:43.305351 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:43.338031 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1435/2917] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:58:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:58:43.349769 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:58:43.351364 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:58:43.367888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:43.420746 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de0bc5-5197-4835-b973-fcb8f3518a8e admin admin] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:58:43.424888 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a2388c351814d5193403a4b0d00b0af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:58:43.425564 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] REQUEST_METHOD: `GET` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:58:43.425849 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:58:43.426122 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:58:43.434776 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:58:43.436245 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb7e44-8036-4585-af58-b918a6d26ea8 service glance] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:58:43.445961 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1483/2918] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:58:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:58:53.335142 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:58:53.441916 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=21 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:59:15.234876 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:59:15.296641 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdaeaf-2e36-4247-a823-8ffc58e9d0c5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:59:15.301132 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:59:15.301987 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:59:15.302269 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:59:15.302544 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] PATH_INFO: `/v3/users/f57c7a722fb54142a90a368952b1368a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:59:15.303724 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57c7a722fb54142a90a368952b1368a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:59:15.306514 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:59:15.351167 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57c7a722fb54142a90a368952b1368a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.351859 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57c7a722fb54142a90a368952b1368a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.363392 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57c7a722fb54142a90a368952b1368a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.370662 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57c7a722fb54142a90a368952b1368a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.371287 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57c7a722fb54142a90a368952b1368a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.384935 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1436/2919] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jul 20 15:59:15 2022] DELETE /identity/v3/users/f57c7a722fb54142a90a368952b1368a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:59:15.389585 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] REQUEST_METHOD: `POST` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:59:15.390059 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:59:15.390180 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:59:15.406434 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.auth.core [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] MFA Rules not processed for user `4fc861f9b4f146eca38f05771344d34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:59:15.407598 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:59:15.465756 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1484/2920] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jul 20 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:59:15.475377 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:59:15.476824 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:59:15.533918 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c61ba5-dd1d-42c7-9a67-ec1633b1e04c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:59:15.537568 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29a153f06f9e4b459a413f2f936c1f9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:59:15.538212 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:59:15.538493 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:59:15.538766 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:59:15.547652 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:59:15.549097 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:59:15.559064 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1437/2921] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 15:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:59:15.832225 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] Authenticating user token {{(pid=80208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:59:15.833703 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.fernet_utils [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:59:15.846866 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824b0b1-6420-4aaa-826d-5ec86a15f5f5 None None] Validating token access rules against request {{(pid=80208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:59:15.852229 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc861f9b4f146eca38f05771344d34d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498a38cb72f746aabe0439ba1e952a81', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:59:15.853098 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:59:15.853428 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] SCRIPT_NAME: `/identity` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:59:15.853704 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] PATH_INFO: `/v3/projects/ed624cca84614be5addb3b8eb7eaf288` {{(pid=80208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:59:15.854874 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed624cca84614be5addb3b8eb7eaf288)` {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:59:15.856944 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] RBAC: Authorization granted {{(pid=80208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:59:15.908167 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed624cca84614be5addb3b8eb7eaf288', 'request_id': 'req-38a5d169-5872-4d06-803a-b97c6ff7187d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.913501 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed624cca84614be5addb3b8eb7eaf288', 'request_id': 'req-38a5d169-5872-4d06-803a-b97c6ff7187d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.921445 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed624cca84614be5addb3b8eb7eaf288', 'request_id': 'req-38a5d169-5872-4d06-803a-b97c6ff7187d'} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.933237 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] The token cache is being invalidate because project ed624cca84614be5addb3b8eb7eaf288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:59:15.933532 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.934155 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG keystone.notifications [None req-38a5d169-5872-4d06-803a-b97c6ff7187d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:59:15.935673 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: [pid: 80208|app: 0|req: 1485/2922] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jul 20 15:59:15 2022] DELETE /identity/v3/projects/ed624cca84614be5addb3b8eb7eaf288 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:59:25.554477 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:59:25.929559 dv-cinder-848759-3-iscsi devstack@keystone.service[80208]: DEBUG dbcounter [-] [80208] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 16:00:53.136102 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 16:00:53.137488 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 16:00:53.208080 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b2d5-50dd-4da9-badb-c5a13157f013 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 16:00:53.265597 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6b54079a045dd9a526fafe8de8bbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f24b63df63c344e68a24f6f2876f92e5', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 16:00:53.266242 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 16:00:53.266524 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 16:00:53.266798 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 16:00:53.268596 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 16:00:53.280025 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 16:00:53.281583 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f1ab4-274d-4ce7-9611-f6b343ca6844 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 16:00:53.291950 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1438/2923] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jul 20 16:00:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 16:01:03.287727 dv-cinder-848759-3-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=58 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}